Safely Evolving Your Schema

One of the biggest fears in application development is changing your data schema. In traditional systems, a simple change—like adding or removing a field—can be a destructive operation that requires downtime, complex migration scripts, and risks data loss. FoxNose solves this with a core design principle: schema changes are always non-destructive.

When you publish a new schema version, your existing content is never automatically changed, deleted, or migrated. This guide explains how this versioning system works and how you can use it to evolve your schema safely and predictably.

How Non-Destructive Migrations Work

FoxNose's approach is built on a simple promise: your live data is sacred.

• Existing Content is Immutable: When you publish a new schema, existing resources remain untouched, continuing to conform to the schema version they were created with.
• New Content Uses the New Schema: Any new resources you create will use the new, published schema version.
• You Control the Migration: This design gives you full control over how and when you migrate old content to the new structure. No unexpected data loss, no forced downtime.
• Applies to Locales Too: This principle also applies when adding or removing locales. Deleting a locale only hides it from the API; the translated content is preserved internally unless you explicitly remove it.

Understanding the Impact of Schema Changes

Here’s how different types of schema changes affect your content and API responses.

Adding a New Field

When you add a new field (e.g., author) to a schema and publish it:

• Existing resources will not have this field. API calls for old resources won't return it.
• New resources will be created with the new field.
• To add the field to an old resource, you simply edit and re-save it, at which point you can (or must, if required) provide a value for the new field.

Removing a Field

When you remove a field (e.g., legacy_id) from a schema and publish it:

• The field disappears from the editor in the dashboard.
• The Flux API stops returning the field in standard queries. Your applications will no longer see it.
• However, the data still exists in storage. You can retrieve it using the ?raw=true parameter in your API call, which is useful for debugging or data recovery.

Changing a Field's Properties

Changing a property like searchable or vectorizable only affects new or updated content.

• The Pain Point: You mark an existing body field as vectorizable and expect semantic search to work immediately on all your old content.
• The Reality: It won't. FoxNose only generates embeddings when a resource is saved. To enable semantic search on existing content, you must re-save those resources to trigger the embedding process.

Practical Migration Strategies

Strategy 1: Additive Changes (Safest)

The simplest and safest strategy is to only add new fields and never remove or rename existing ones. Your application code should be written to handle cases where older resources may not have the new fields.

Strategy 2: Gradual Migration (for Renaming/Restructuring)

When you need to rename or restructure a field (e.g., changing summary to description):

1. Add New Field: In v2 of your schema, add the new description field alongside the old summary field.
2. Update Your App: Your application code should start writing to both fields, or prefer writing to the new one.
3. Backfill Data: Run a script that reads each resource, copies the content from summary to description, and re-saves the resource.
4. Deprecate Old Field: In v3 of your schema, remove the summary field.
5. Finalize App Code: Update your application to only read from the new description field.

Strategy 3: Bulk Re-Save for Indexing

When you make a field searchable or vectorizable, you need to trigger a re-index for all existing content. You can do this with a simple script using the Management API.

# Pseudocode to re-save all resources in a folder
# This triggers re-indexing and embedding generation.
for resource in get_all_resources(folder_id):
    # Creating a new revision with the same data is enough
    create_new_revision(resource)

Common Pitfalls to Avoid

• Problem: Reusing a field key with a different data type.

  • Example: Changing price from a string ("19.99") to a number (19.99).
  • Why it's bad: Your application will receive mixed data types, leading to errors.
  • Solution: Always create a new field key (price_numeric) and migrate the data.

• Problem: Making an optional field required.

  • Example: Changing an optional author field to be required.
  • Why it's bad: You can no longer edit any old resources that don't have an author value, as they will fail the new validation rule.
  • Solution: Backfill the data to add an author to all existing resources before making the field required.