Redis Migration Walkthrough

This is a complete, plug-and-play walkthrough that walks you through a Redis migration workflow. All commands use localhost and can be copied and executed directly without modification.

For production DNS, private routing, load balancer, sidecar, and VPN placement patterns, read Deployment And DNS Architecture before adapting this local walkthrough.

Prerequisites

Before running this walkthrough, ensure you have:

  1. Eve running on localhost:8000
  2. Two Redis instances:

- Source Redis on localhost:6378

- Target Redis on localhost:6377

  1. The organization creation token set as environment variable:
bash
export EDEN_NEW_ORG_TOKEN="your_org_creation_token"

Step 1: Create Organization

Create a new organization with a super admin user:

bash
curl http://localhost:8000/api/v1/new \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $EDEN_NEW_ORG_TOKEN" \
  -d '{
    "id": "TestOrg",
    "description": "test organization",
    "super_admins": [
      {
        "username": "admin",
        "password": "password",
        "description": null
      }
    ]
  }'

Expected Response:

json
{
  "id": "TestOrg",
  "uuid": "550e8400-e29b-41d4-a716-446655440000"
}

Step 2: Login

Login with basic authentication and save the token:

bash
curl http://localhost:8000/api/v1/auth/login \
  -u admin:password \
  -H "X-Org-Id: TestOrg" \
  -X POST

Expected Response:

json
{
  "token": "eyJhbGciOiJIUzI1NiIs..."
}

Save the token for subsequent requests:

bash
export TOKEN="eyJhbGciOiJIUzI1NiIs..."

Step 3: Create Source Redis Endpoint

Create the first Redis endpoint (source database on port 6378):

bash
curl http://localhost:8000/api/v1/endpoints \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "endpoint": "redis_test1",
    "kind": "redis",
    "config": {
      "read_conn": null,
      "write_conn": {
        "host": "localhost",
        "port": 6378,
        "password": null,
        "tls": false
      }
    }
  }'

Expected Response:

json
{
  "status": "success",
  "data": {
    "id": "redis_test1",
    "uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
  }
}

Save the UUID for later use:

bash
export REDIS1_UUID="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"

Step 4: Create Target Redis Endpoint

Create the second Redis endpoint (target database on port 6377):

bash
curl http://localhost:8000/api/v1/endpoints \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "endpoint": "redis_test2",
    "kind": "redis",
    "config": {
      "read_conn": null,
      "write_conn": {
        "host": "localhost",
        "port": 6377,
        "password": null,
        "tls": false
      }
    }
  }'

Expected Response:

json
{
  "status": "success",
  "data": {
    "id": "redis_test2",
    "uuid": "ffffffff-1111-2222-3333-444444444444"
  }
}

Save the UUID:

bash
export REDIS2_UUID="ffffffff-1111-2222-3333-444444444444"

Step 5: Create an Interlay

Create an interlay that initially routes traffic to the source endpoint. The interlay exposes port 6366 for client connections:

bash
curl http://localhost:8000/api/v1/interlays \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "id": "redis_interlay",
    "endpoint": "'$REDIS1_UUID'",
    "port": 6366,
    "settings": {},
    "tls": false
  }'

Expected Response:

json
{
  "status": "success",
  "data": {
    "id": "redis_interlay",
    "uuid": "11111111-2222-3333-4444-555555555555"
  }
}

Step 6: Write Data to Source Redis

Write some test data to the source Redis through the endpoint:

bash
curl http://localhost:8000/api/v1/endpoints/$REDIS1_UUID/write \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "request": {
      "type": "Set",
      "key": "test_key",
      "value": "hello_world"
    }
  }'

Expected Response:

json
{
  "kind": "redis",
  "data": [{ "Resp3": [43, 79, 75, 13, 10] }]
}

The response contains RESP3 protocol bytes representing "OK".

Verify the data was written:

bash
curl http://localhost:8000/api/v1/endpoints/$REDIS1_UUID/read \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "request": {
      "type": "Get",
      "key": "test_key"
    }
  }'

Expected Response:

json
{
  "kind": "redis",
  "data": [
    {
      "Resp3": [
        36, 49, 49, 13, 10, 104, 101, 108, 108, 111, 95, 119, 111, 114, 108,
        100, 13, 10
      ]
    }
  ]
}

The response contains RESP3 protocol bytes representing "hello_world".

Step 7: Create a Migration

Create a new migration that will move data from source to target:

bash
curl http://localhost:8000/api/v1/migrations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "id": "test_migration",
    "description": "Test migration from redis_test1 to redis_test2",
    "strategy": {
      "type": "big_bang",
      "durability": true
    }
  }'

Expected Response:

json
{
  "status": "success",
  "data": {
    "id": "test_migration",
    "uuid": "66666666-7777-8888-9999-aaaaaaaaaaaa"
  }
}

Step 8: Add Interlay to Migration

Link the interlay to the migration with traffic routing rules:

bash
curl http://localhost:8000/api/v1/migrations/test_migration/interlay/redis_interlay \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "id": "migration_relay",
    "endpoint": "redis_test2",
    "description": "Redis migration interlay",
    "migration_strategy": {
      "type": "big_bang",
      "durability": true
    },
    "migration_data": {
      "Scan": {
        "replace": "None"
      }
    },
    "migration_rules": {
      "traffic": {
        "read": "Replicated",
        "write": "New"
      },
      "error": "DoNothing",
      "rollback": "Ignore",
      "completion": {
        "milestone": "Immediate",
        "require_manual_approval": false
      }
    }
  }'

Expected Response:

json
{
  "status": "success",
  "data": "added Interlay to migration"
}

Step 9: Start the Migration

Execute the migration to transfer data and switch traffic:

bash
curl http://localhost:8000/api/v1/migrations/test_migration/migrate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -X POST

Expected Response:

json
{
  "status": "success",
  "data": "Migration started"
}

Step 10: Verify Migration

Check that data has been migrated to the target Redis:

bash
curl http://localhost:8000/api/v1/endpoints/$REDIS2_UUID/read \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "request": {
      "type": "Get",
      "key": "test_key"
    }
  }'

Expected Response:

json
{
  "kind": "redis",
  "data": [
    {
      "Resp3": [
        36, 49, 49, 13, 10, 104, 101, 108, 108, 111, 95, 119, 111, 114, 108,
        100, 13, 10
      ]
    }
  ]
}

The response contains RESP3 protocol bytes representing "hello_world".

Complete Walkthrough Script

Here's the entire walkthrough as a single script you can run:

bash
#!/bin/bash
set -e

# Configuration
HOST="localhost:8000"
EDEN_NEW_ORG_TOKEN="${EDEN_NEW_ORG_TOKEN:-your_org_creation_token}"

echo "=== Step 1: Create Organization ==="
curl -s http://$HOST/api/v1/new \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $EDEN_NEW_ORG_TOKEN" \
  -d '{
    "id": "TestOrg",
    "description": "test organization",
    "super_admins": [{"username": "admin", "password": "password", "description": null}]
  }' | jq .

echo -e "\n=== Step 2: Login ==="
TOKEN=$(curl -s http://$HOST/api/v1/auth/login \
  -u admin:password \
  -H "X-Org-Id: TestOrg" \
  -X POST | jq -r '.token')
echo "Token: ${TOKEN:0:50}..."

echo -e "\n=== Step 3: Create Source Redis Endpoint ==="
REDIS1_RESPONSE=$(curl -s http://$HOST/api/v1/endpoints \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "endpoint": "redis_test1",
    "kind": "redis",
    "config": {"read_conn": null, "write_conn": {"host": "localhost", "port": 6378, "password": null, "tls": false}}
  }')
echo $REDIS1_RESPONSE | jq .
REDIS1_UUID=$(echo $REDIS1_RESPONSE | jq -r '.data.uuid')

echo -e "\n=== Step 4: Create Target Redis Endpoint ==="
REDIS2_RESPONSE=$(curl -s http://$HOST/api/v1/endpoints \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "endpoint": "redis_test2",
    "kind": "redis",
    "config": {"read_conn": null, "write_conn": {"host": "localhost", "port": 6377, "password": null, "tls": false}}
  }')
echo $REDIS2_RESPONSE | jq .
REDIS2_UUID=$(echo $REDIS2_RESPONSE | jq -r '.data.uuid')

echo -e "\n=== Step 5: Create Interlay ==="
curl -s http://$HOST/api/v1/interlays \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d "{
    \"id\": \"redis_interlay\",
    \"endpoint\": \"$REDIS1_UUID\",
    \"port\": 6366,
    \"settings\": {},
    \"tls\": false
  }" | jq .

echo -e "\n=== Step 6: Write Test Data ==="
curl -s http://$HOST/api/v1/endpoints/$REDIS1_UUID/write \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"request": {"type": "Set", "key": "test_key", "value": "hello_world"}}' | jq .

echo -e "\n=== Step 7: Create Migration ==="
curl -s http://$HOST/api/v1/migrations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "id": "test_migration",
    "description": "Test migration from redis_test1 to redis_test2",
    "strategy": {"type": "big_bang", "durability": true}
  }' | jq .

echo -e "\n=== Step 8: Add Interlay to Migration ==="
curl -s http://$HOST/api/v1/migrations/test_migration/interlay/redis_interlay \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "id": "migration_relay",
    "endpoint": "redis_test2",
    "description": "Redis migration interlay",
    "migration_strategy": {"type": "big_bang", "durability": true},
    "migration_data": {"Scan": {"replace": "None"}},
    "migration_rules": {
      "traffic": {"read": "Replicated", "write": "New"},
      "error": "DoNothing",
      "rollback": "Ignore",
      "completion": {"milestone": "Immediate", "require_manual_approval": false}
    }
  }' | jq .

echo -e "\n=== Step 9: Start Migration ==="
curl -s http://$HOST/api/v1/migrations/test_migration/migrate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -X POST | jq .

echo -e "\n=== Step 10: Verify Data on Target ==="
sleep 2  # Wait for migration to complete
curl -s http://$HOST/api/v1/endpoints/$REDIS2_UUID/read \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"request": {"type": "Get", "key": "test_key"}}' | jq .

echo -e "\n=== Walkthrough Complete ==="

Save this as redis_migration_walkthrough.sh, make it executable with chmod +x redis_migration_walkthrough.sh, and run it.

Traffic Routing Rules

During migration, you can configure how traffic is routed:

RuleRead BehaviorWrite Behavior
OldRead from sourceWrite to source only
NewRead from targetWrite to target only
ReplicatedRead from both (compare)N/A
BothN/AWrite to both endpoints

Cleanup

To clean up after the walkthrough:

bash
# Delete the migration
curl http://localhost:8000/api/v1/migrations/test_migration \
  -H "Authorization: Bearer $TOKEN" \
  -X DELETE

# Delete the interlay
curl http://localhost:8000/api/v1/interlays/redis_interlay \
  -H "Authorization: Bearer $TOKEN" \
  -X DELETE

# Delete the endpoints
curl http://localhost:8000/api/v1/endpoints/redis_test1 \
  -H "Authorization: Bearer $TOKEN" \
  -X DELETE

curl http://localhost:8000/api/v1/endpoints/redis_test2 \
  -H "Authorization: Bearer $TOKEN" \
  -X DELETE

# Delete the organization (requires super admin)
curl http://localhost:8000/api/v1/organizations \
  -H "Authorization: Bearer $TOKEN" \
  -X DELETE
Last updated: July 10, 2026