devkuma – Redis

Redis Overview

kc@example.com (kc kim) — Mon, 24 May 2021 10:12:26 +0900

Overview

Redis stands for Remote Dictionary Server. It is an open source, non-relational database management system for storing and managing unstructured data in a key-value structure. It was first developed by Salvatore Sanfilippo in 2009. Since 2015, it has been supported by Redis Labs. Redis is used freely in many services.

Pareto principle

The Pareto principle says that 80% of the phenomena that occur in society are caused by 20% of the causes. This also fits website access patterns.

It is estimated that 80% of Internet traffic is access to only 20% of sites, and caching the data from these 20% of websites can dramatically improve efficiency.

Textbook for Infrastructure Engineers - Network Edition, 2017, Gilbut

Therefore, storing commonly used data in Redis as a cache can be an efficient way to use resources.

Redis features

Redis is a key-value store NoSQL DB.
- Supports a key-value structure for simple strings
- Supports writing binary data
It operates as a single thread.
- It guarantees operation atomicity by operating as a single thread.
It supports various data structures.
- Collection support: List, Set, Sorted Set, Hash, and other data structures
High speed
- It provides performance around 100,000 QPS.
Disk storage
- Data can be stored persistently on disk.
Replication
- Supports a master/slave structure that can replicate data to other nodes.
Sharding support
- Supports data sharding through Redis Cluster.
Publish/Subscribe support
- Can be used as a message broker
Expire time
- You can set an expiration time on a key so old data is automatically deleted.
Recovery is possible through data snapshots or AOF logs, so some persistence is guaranteed.
In Java Spring, it is often used for session management and caching.

Why are collections important?

Redis is an in-memory database. That means all data is stored and queried in memory. It is much faster than traditional relational databases such as Oracle and MySQL because memory access is faster than disk access. However, speed is only one of Redis’s characteristics. The biggest difference from other in-memory databases such as Memcached is that it supports various data structures. Redis stores various data structures in key-value form as shown below.

Redis basically provides String, Bitmap, Hash, List, Set, and Sorted Set, and as versions have advanced, it now also supports types such as Geospatial Index, HyperLogLog, and Stream.

Why is providing many data structures important? The reason is development convenience and difficulty.

For example, if you implement a real-time ranking server with an RDBMS, you need to store data in the DB, sort it by the stored SCORE value, and read it again. As the number of records increases, this process becomes slower because it uses disk. You could write code directly to process data on the server with an in-memory approach, but using Redis Sorted Set is faster and simpler.

Redis persistence

Redis uses RDB (Snapshot) and AOF (Append Only File) methods to store data on disk.
Both methods internally use the fork() function, so sufficient memory is required.

RDB (Snapshot)
- A method of moving data in memory to disk.
- Because snapshots capture the memory state as it is, they are useful for backup and recovery at a specific point in time, and they can load data into memory faster than AOF.
- If the server goes down, data changed between backed-up snapshots is lost.
- There are two methods.
  - SAVE: a blocking method that stops all Redis operations sequentially and stores the snapshot at that time on disk.
  - BGSAVE: a non-blocking method that stores the memory snapshot at the time of execution on disk through a separate process.
AOF (Append Only File)
- A method of recording all Redis write/update operations as a log file.
- When the server starts, operations are replayed sequentially to recover data.
- Because it records every operation as it runs, logs can be kept up to the current point.
- Because it only appends to the log file, write speed is fast and data loss does not occur even if the server goes down.
- Because logs are recorded for all operations, the amount of data is very large.
- When the server restarts, all operations must be executed again, so restart speed is slow.

Using Redis in the cloud

If you use Amazon Web Services or Microsoft Azure for application development, each cloud service provides a Redis service, so you can use it.

Cloud service	Service name
Amazon Web Services	Amazon ElastiCache for Redis
Microsoft Azure	Azure Redis Cache

Of course, you can also install Redis directly on each cloud service’s compute instances, but considering management cost, choosing these services is recommended.

References

Redis | Wikipedia
[redisgate][http://redisgate.kr/]

Installing Redis with Docker

kc@example.com (kc kim) — Fri, 31 Mar 2023 17:46:26 +0900

Installing Redis with Docker

If you want a simple setup for learning or experimentation, using Docker to install Redis is recommended.

Downloading the Redis Docker image

Run the following command to download Redis Docker. This example uses the alpine version.

If you want another version tag, refer to https://hub.docker.com/_/redis/tags.

% docker pull redis:alpine

After downloading, check the image and you can see the list as follows.

% docker images
REPOSITORY         TAG       IMAGE ID       CREATED        SIZE
redis              alpine    a4cf5af74f5e   2 days ago     30.4MB

Creating and running a Redis Docker container

Run the following command.

% docker run -d -p 6379:6379 --name=redis redis:alpine

-p: specify the port to expose on the host
--name: specify the container name

After running it, check the running container.

% docker container ls
CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                    NAMES
1df277cb625c   redis:alpine   "docker-entrypoint.s…"   30 seconds ago   Up 29 seconds   0.0.0.0:6379->6379/tcp   redis

Connecting to Redis Docker

Connecting inside the Redis Docker container

Before connecting to Redis, connect inside the Docker container where Redis is installed.

% docker exec -it redis /bin/bash

If you see an error message like the following, try connecting with /bin/sh.

OCI runtime exec failed: exec failed: unable to start container process: exec: "/bin/bash": stat /bin/bash: no such file or directory: unknown

% docker exec -it redis /bin/sh

Connecting to Redis

Once connected inside the container, connect to Redis. Use the redis-cli command to connect to Redis.

/data # redis-cli
127.0.0.1:6379>

When connected, the prompt changes to IP:port> as shown above.

To check Redis information, enter info as follows.

127.0.0.1:6379> info
# Server
redis_version:7.0.10

... omitted ...

Press Ctrl+C to exit.

Redis redis-cli

kc@example.com (kc kim) — Fri, 10 Mar 2023 14:39:26 +0900

Installing redis-cli

The following shows how to install it with Homebrew in a macOS environment.

brew tap aoki/redis-cli
brew update && brew doctor
brew install redis-cli

github.com/aoki/homebrew-redis-cli

Server control

Connecting with redis-cli

If the host name and port number are omitted, it connects to localhost on port 6379.

$ redis-cli

Remote access can be done as follows.

$ redis-cli -h #{host name} -p #{port number}

Other main options are as follows.

-n: database number
-a: password
-s: socket
-u: server URL and other connection options can be set

Querying server information

redis-cli info

> info
# Server
redis_version:6.0.9
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:ffd199d8341c2d8f
redis_mode:standalone
... omitted ....

Querying connected clients

> client list

Disconnecting a connected client

> client kill 10.0.0.8:33333

Dumping all requests received after running the command

redis-cli monitor

> monitor
OK

Key operations

Setting a key and value

set {key name} {value}

> set key01 value01
OK

Querying a value by key

get {key name}

> get key01
value01

Querying key list

> keys *
key01

Deleting a key

> del key01
1

Checking whether a key exists, where 1 means true

exists {key name} {value}

> exists key01
0

Setting data only if the key does not already exist

setnx {key name} {value}

> setnx key01 value02
1
> setnx key01 value02
0

If the result is 0, it means the key exists and the data was not updated.

Renaming an existing key

rename {old key name} {new key name}

> rename key01 key02 
OK
> rename key03 key04
ERR no such key

Returning the number of keys in the current database

> dbsize
2

Moving a specific key to another database

> move key01 1
1

Selecting a DB number

> select 1
OK

Deleting all keys in the current database

> flushdb
OK

Deleting all keys in all databases

> flushall
OK

Regular expressions

The regular expressions available in Redis are only the following.

[]: one of the characters inside the brackets
*: any string
?: a single character

Examples

# Query all Key lists.
$ keys *

# Query keys that start with a number.
$ keys [0-9]*

Multiple Key

Setting multiple key values together

> mset key01 value01 key02 value02 key03 value03 
OK

Querying multiple key values at once

> mget key01 key02 key03
value01
value02
value03

expire

Setting a key with an expiration period

The following example sets it to 60 seconds.

> setex key01 60 value01 
OK

Setting expiration on an existing key

> expire key02 30

Math

Adding 1 to a key value

The following example sets key02 to 1 and increments it by 1.

> set key02 1 
OK
> incr key02 
2

Adding a specified number to a key value

> incrby key02 100 
102

Subtracting 1 from a key value

> decr key02 
101

Subtracting a specified number from a key value

> decrby key02 51 
50

List

Adding values to the head of a List

Multiple values can be specified separated by spaces.

> lpush list01 value01
1

Adding values to the end of a List

Multiple values can be specified separated by spaces.

> rpush list01 value02
2

Returning the number of List elements

> llen list01
2

Returning the element at a specified index in a List

> lindex list01 0
value01

Returning elements from a specified start index to end index in a List

> lrange list01 0 1
value01
value02

Changing the element at a specified index in a List to a specified value

> lset list01 1 value03
OK
> lindex list01 1
value03

Returning and deleting the first element of a List

> lpop list01
value01

Returning and deleting the last element of a List

> rpop list01
value03

Trimming a List to elements from a specified start index to end index

> ltrim list01 1 3
OK

Hash

This is a data format managed by a parent Key, child Key (field), and value. For example, if you use the parent Key as a product ID and child keys to store product attribute information, you can use it with a way of thinking close to RDB data management.

Adding a value to a specified Hash field

> hset hash01 field01 value01
(integer) 1

Returning the value of a specified Hash field

> hget hash01 field01
"value01"

Adding multiple fields to a Hash

> hmset hash01 field02 value02 field03 value03
OK

Returning the values of multiple Hash fields

> hmget hash01 field01 field02 field03
value01
value02
value03

Returning all Hash fields

> hkeys hash01
field01
field02
field03

Returning all values of Hash fields

> hvals hash01
value01
value02
value03

Adding a specified number to the value of a specified Hash field

> hincrby hash01 field04 100
101

Checking whether a specified Hash field exists

> hexists hash01 field01
1

Returning the number of Hash fields

> hlen hash01
4

Removing a specified Hash field

> hdel hash01 field04
1

Other operations

How to search for keys when keys cannot be used

scan 0 match devkuma-* count 1000

Key names can be searched with wildcards, and devkuma-* searches for keys that start with devkuma-.

Querying and Storing Redis Data Structures

kc@example.com (kc kim) — Fri, 31 Mar 2023 17:46:26 +0900

Data structures (Collection)

Redis provides various data structures.

In Redis, the basic type for data representation consists of one Key and one or more Field/Element values.

Key
- Stores an ASCII value.
Value (Field/Element)
- Basically stores String data.
- Stores Container-type data.
Container type
- Includes Hash, List, Set, Sorted Set, and others.

Strings

String is the most basic data type for Redis values. It stores characters, numbers, and other values. When storing, there is no separate distinction between numbers and characters.
Redis Strings are binary safe. This means a Redis string can contain any kind of data, such as a JPEG image or a serialized Ruby object.

A String value can have a maximum length of 512 MB.

Redis commands for Strings

Let’s store and query data in a key-value structure.

Command	Syntax	Description
`set`	`set key`	Stores a key-value pair.
`get`	`get key`	Queries the value for the key.
`del`	`del key`	Deletes the key.

127.0.0.1:6379> set devkuma 1000
OK
127.0.0.1:6379> get devkuma
"1000"
127.0.0.1:6379> del devkuma
(integer) 1
127.0.0.1:6379> get devkuma
(nil)
127.0.0.1:6379>

Lists

A List is simply a String list ordered by insertion order. You can think of the String type as becoming an array structure.
New elements can be added to the front or back of a list. It is said to support up to about 4 billion elements.

Command	Syntax	Description
`lpush`	`LPUSH key element [element ...]`	Left push. Stores data from the left, with the index starting at 0.
`lpop`	`LPOP key`	Left pop. Extracts data from index 0 of the list.
`rpush`	`RPUSH key element [element ...]`	Right push. Stores data from the right, meaning from the last index.
`rpop`	`RPOP key`	Right pop. Extracts data from the last index of the list.
`lrange`	`LRANGE key start stop`	Extracts list data from start to end. If end is declared as -1 in lrange, all data in the list is extracted.

The LPUSH command adds a new element before the head, and RPUSH adds it at the end. If executed on an empty key, a new list is created and added.

127.0.0.1:6379> lpush fruit apple
(integer) 1
127.0.0.1:6379> lpush fruit banana
(integer) 2
127.0.0.1:6379> lpush fruit orange
(integer) 3
127.0.0.1:6379> lrange fruit 0 2
1) "orange"
2) "banana"
3) "apple"
127.0.0.1:6379>

Sets

A Set is an unordered String collection. It is like a List type without indexes. Values are not duplicated. It can be processed faster than Lists.

Because data can be extracted randomly, it may be usable for areas such as machine learning.

Command	Syntax	Description
`sadd`	`SADD key member [member...]`	Adds members to the set key.
`srem`	`SREM key member [member...]`	Deletes members from the set key.
`smembers`	`SMEMBERS key`	Queries all members of the set key.

Add elements to key animal with the sadd command.

127.0.0.1:6379> sadd animal dog
(integer) 1
127.0.0.1:6379> smembers animal
1) "dog"
127.0.0.1:6379> sadd animal cat cow
(integer) 2
127.0.0.1:6379> smembers animal
1) "dog"
2) "cat"
3) "cow"

Delete an element from key animal with the srem command.

127.0.0.1:6379> srem animal cat
(integer) 1
127.0.0.1:6379> smembers animal
1) "dog"
2) "cow"
127.0.0.1:6379>

Use the del command to delete everything.

127.0.0.1:6379> del animal
(integer) 1
127.0.0.1:6379> smembers animal
(empty array)
127.0.0.1:6379>

Sorted Sets

A Sorted Set is a set of non-repeating Strings similar to a Redis Set. It is a Set type with the concept of score added. Values are not duplicated here either. It can be sorted by score or processed with threshold values on scores. It seems usable for real-time ranking displays in games.

Hashes

A Hash consists of multiple field-value pairs under one key. It is a Set with the concept of fields that can be specified as strings.

Every hash can store up to 2^32 - 1 field-value pairs, or more than about 4 billion.

Command	Syntax	Description
`hset`	`HSET key field [field value...]`	Sets one or more values.
`hget`	`HGET key field`	Gets one or more values.

Bitmaps and HyperLogLogs

Redis also supports Bitmaps and HyperLogLogs, data types that are actually based on the String base type but have their own meanings.

References

Redis data types | Redis

Redis Command SETEX key Seconds value

kc@example.com (kc kim) — Fri, 31 Mar 2023 17:46:26 +0900

SETEX

The SETEX command sets a string value with a key and seconds. The data is deleted after the specified number of seconds.
It is useful for storing data with an expiration time.

Available version: version 2.0.0 and later
Logical processing time complexity: O(1)

It behaves the same as the following two commands.

SET mykey value
EXPIRE mykey seconds

SETEX usage

The usage is as follows.

SETEX [key] [seconds] [value]

When creating a key to store a string value, you can also set a timeout. The key is deleted after the timeout period has passed. It has the same effect as the following two commands.

Data is deleted after the specified time: specify in seconds. Data is deleted after the specified number of seconds. It is useful for storing data with an expiration time. The usage is SETEX key seconds value.

Example Command> setex key 5 value Result> OK Command> ttl key Result> 2 shows the remaining time in seconds Command> get key Result> (nil) data was deleted after 5 seconds View animation

The TTL command shows the remaining time in seconds.