Heterogeneous Migration Planning

Heterogeneous migrations move data and live traffic between different endpoint families, such as PostgreSQL and MongoDB. They require two plans before execution can start:

• a schema mapping for historical data movement;
• a request mapping for observed reads, writes, stored procedures, and response shapes.

The migration constructor does not accept the final heterogeneous scan mapping. Eve discovers the source shape during planning, then an operator or planning agent patches schema mappings and request mappings until planning is complete.

Required Flow

1. Create source and target endpoints.
2. Create the migration shell with POST /api/v1/migrations.
3. Attach source and target API entries and interlays.
4. Run analysis and compatibility.
5. Start query seeding while representative live source traffic is flowing.
6. Run planning to discover source schema and source keys.
7. Patch schema mapping with either a key-value map or a full route mapping.
8. Patch request mappings for every observed read, write, and procedure shape.
9. Validate and finalize planning.
10. Run execution preflight.
11. Start execution with POST /api/v1/migrations/{migrationId}/migrate.

Schema Mapping

Schema mapping uses stable source keys:

• PostgreSQL and MySQL object keys use schema.table.
• MongoDB object keys use database.collection.
• PostgreSQL and MySQL fields use column names.
• MongoDB fields use dotted document paths.

Every source object and field that can carry data must be mapped or explicitly blocked. Missing mappings keep the migration in planning.

The source and target endpoint kinds are route-level facts. In source_key_map, each entry names the target object; Eve infers the target kind from the migration target endpoint. The older { "kind": "...", "name": "..." } target object form is still accepted for compatibility, but it is not required in patch requests.

{
  "source_key_map": {
    "public.users": {
      "target": "app.users"
    },
    "public.users.id": {
      "target": "app.users",
      "target_field": "_id",
      "target_type": "text",
      "representation": "scalar"
    },
    "public.users.email": {
      "target": "app.users",
      "target_field": "profile.email",
      "target_type": "text",
      "representation": "scalar"
    }
  },
  "limits": {
    "max_batch_bytes": 33554432,
    "max_batch_rows": 4093
  }
}

Patch schema mapping:

curl -sS "$EDEN/migrations/$MIGRATION_ID/planning/schema-mapping/missing" \
  -H "$AUTH_HEADER"

curl -sS "$EDEN/migrations/$MIGRATION_ID/planning/target-schema-plan" \
  -H "$AUTH_HEADER"

curl -sS -X PATCH "$EDEN/migrations/$MIGRATION_ID/planning/schema-mapping" \
  -H "$AUTH_HEADER" \
  -H "Content-Type: application/json" \
  -d @schema-key-map.json

Dual Mapping Example: PostgreSQL And MongoDB

For heterogeneous migrations, "dual mapping" means two separate mapping surfaces are required:

• schema mapping: how source data lands in the target model during historical copy;
• request mapping: how observed source requests become target requests, and how target responses are converted back to the source response shape when target reads are enabled.

Reverse migration is not inferred automatically. PostgreSQL -> MongoDB and MongoDB -> PostgreSQL use related ideas, but each direction has its own source keys, target keys, conflict rules, and response templates.

Direction Matrix

Coverage Matrix

Operation Matrix

PostgreSQL -> MongoDB Schema Mapping

This direction treats PostgreSQL tables and columns as source keys and maps them into MongoDB collections and dotted document paths. Relational relationships can be preserved as references, embedded documents, or both, but the choice must be explicit.

{
  "source_kind": "postgres",
  "target_kind": "mongo",
  "source_key_map": {
    "public.customers": {
      "target": "commerce.customers",
      "identity": "public.customers.id"
    },
    "public.customers.id": {
      "target": "commerce.customers",
      "target_field": "_id",
      "target_type": "objectId_or_string",
      "representation": "scalar"
    },
    "public.customers.email": {
      "target": "commerce.customers",
      "target_field": "profile.email",
      "target_type": "string",
      "representation": "scalar"
    },
    "public.customers.created_at": {
      "target": "commerce.customers",
      "target_field": "timestamps.createdAt",
      "target_type": "date",
      "representation": "scalar"
    },
    "public.customers.metadata": {
      "target": "commerce.customers",
      "target_field": "metadata",
      "target_type": "document",
      "representation": "json"
    },
    "public.orders": {
      "target": "commerce.orders",
      "identity": "public.orders.id"
    },
    "public.orders.customer_id": {
      "target": "commerce.orders",
      "target_field": "customer._id",
      "target_type": "objectId_or_string",
      "representation": "scalar",
      "relationship": {
        "source_fk": "public.orders.customer_id",
        "references": "public.customers.id"
      }
    }
  },
  "index_plan": [
    {
      "target": "commerce.customers",
      "keys": { "profile.email": 1 },
      "unique": true
    },
    {
      "target": "commerce.orders",
      "keys": { "customer._id": 1, "createdAt": -1 }
    }
  ]
}

PostgreSQL -> MongoDB Live Mapping

For live dual-write, the observed SQL shape is the key. The translation extracts SQL parameters and inserts them into a precompiled MongoDB command template. If reads are shifted to MongoDB, the response template must return the same shape the PostgreSQL client expects.

{
  "query_key": "pg:public.customers:insert:v1",
  "source": {
    "kind": "postgres",
    "operation": "write",
    "normalized": "INSERT INTO public.customers (id,email,created_at,metadata) VALUES ($1,$2,$3,$4) RETURNING id,email,created_at"
  },
  "target": {
    "operation": "insertOne",
    "database": "commerce",
    "collection": "customers",
    "template": {
      "document": {
        "_id": "$1",
        "profile": { "email": "$2" },
        "timestamps": { "createdAt": "$3" },
        "metadata": "$4"
      }
    }
  },
  "response_translation": {
    "source_protocol": "postgres",
    "columns": ["id", "email", "created_at"],
    "values": ["$target.insertedId", "$2", "$3"]
  }
}

MongoDB -> PostgreSQL Schema Mapping

This direction treats MongoDB collection paths as source keys and maps them into tables and columns. MongoDB _id must map to a stable PostgreSQL primary key or an explicit generated key rule.

{
  "source_kind": "mongo",
  "target_kind": "postgres",
  "source_key_map": {
    "commerce.customers": {
      "target": "public.customers",
      "identity": "commerce.customers._id"
    },
    "commerce.customers._id": {
      "target": "public.customers",
      "target_field": "id",
      "target_type": "text",
      "representation": "scalar",
      "primary_key": true
    },
    "commerce.customers.profile.email": {
      "target": "public.customers",
      "target_field": "email",
      "target_type": "text",
      "representation": "scalar"
    },
    "commerce.customers.timestamps.createdAt": {
      "target": "public.customers",
      "target_field": "created_at",
      "target_type": "timestamptz",
      "representation": "scalar"
    },
    "commerce.customers.metadata": {
      "target": "public.customers",
      "target_field": "metadata",
      "target_type": "jsonb",
      "representation": "json"
    }
  },
  "ddl_plan": {
    "public.customers": {
      "primary_key": ["id"],
      "indexes": [
        { "columns": ["email"], "unique": true },
        { "columns": ["created_at"] }
      ]
    }
  }
}

MongoDB -> PostgreSQL Live Mapping

The observed MongoDB command shape is the key. The target template uses SQL parameters, and response translation rebuilds a Mongo-compatible result so the original MongoDB client does not see a protocol or shape change.

{
  "query_key": "mongo:commerce.customers:updateOne:v1",
  "source": {
    "kind": "mongo",
    "operation": "write",
    "normalized": "{ update: 'customers', updates: [{ q: { _id: ? }, u: { $set: { 'profile.email': ? } } }] }"
  },
  "target": {
    "operation": "execute",
    "sql": "UPDATE public.customers SET email = $2 WHERE id = $1",
    "params": [
      "$source.updates[0].q._id",
      "$source.updates[0].u.$set.profile.email"
    ]
  },
  "response_translation": {
    "source_protocol": "mongo",
    "document": {
      "acknowledged": true,
      "matchedCount": "$target.row_count",
      "modifiedCount": "$target.row_count"
    }
  }
}

Dual-Mapping Rules

• Historical copy can use schema mapping only; live migration requires schema mapping plus live request and response mapping.
• A field can be mapped, blocked, or marked lossy only during planning. Lossy mappings must block cutover unless explicitly acknowledged by policy.
• Relationship handling must be explicit: preserve a foreign key, embed the referenced object, store a reference, or split into multiple target writes.
• Generated IDs, UUID formats, timestamps, decimal precision, binary values, arrays, JSON/JSONB, and BSON documents must declare conversion rules.
• Stored procedures and MongoDB aggregation pipelines require explicit templates. They are never considered safely auto-mapped.
• Reverse mappings are separate migration plans. Eve may recommend them from the forward plan, but it does not assume they are correct.

Query Seeding And Request Mapping

Query seeding observes live traffic through the gateway and records normalized query shapes. Each discovered shape records operation class, source endpoint, normalized query or command shape, requesting server IP metadata, and mapping status.

curl -sS -X POST "$EDEN/migrations/$MIGRATION_ID/planning/query-seeding/start" \
  -H "$AUTH_HEADER" \
  -H "Content-Type: application/json" \
  -d '{"duration_secs":3600}'

curl -sS "$EDEN/migrations/$MIGRATION_ID/planning/query-seeding/queries" \
  -H "$AUTH_HEADER"

curl -sS "$EDEN/migrations/$MIGRATION_ID/planning/request-mapping/missing" \
  -H "$AUTH_HEADER"

Patch one request mapping:

curl -sS -X PATCH "$EDEN/migrations/$MIGRATION_ID/planning/request-mapping/query/$QUERY_KEY" \
  -H "$AUTH_HEADER" \
  -H "Content-Type: application/json" \
  -d @query-translation.json

Request mapping must include the target operation template for writes. If target reads are allowed during blue/green, canary, or strangler patterns, the translation must also include response translation so clients continue to receive the original source protocol response shape.

Runtime Query Misses

An unmapped live query is recorded as RuntimeQueryMiss and evaluated through the existing migration error policy. It is not a separate durability configuration.

Operators and agents should use these APIs to inspect and repair misses:

Safety Rules

• DDL must be blocked during heterogeneous execution. New schema changes require another compatibility and planning pass.
• Stored procedures must have explicit request mappings or be blocked.
• Schema mapping is immutable while a migration is running. Schema fixes require canceling and replanning.
• Runtime patching may add only a missing request mapping that uses already mapped schema keys.
• POST /api/v1/migrations/{migrationId}/migrate, target-read activation, and cutover APIs reject migrations that are still in planning.
• LogAndReplay requires durable write-log capture.

Related Docs

• API Stage Map
• Migration API Reference
• Operating Exodus