nextv2.0.0
Run the server

Back up and restore the database

Back up Kraft's database while the server runs, check the backup, restore it, and find the copy Kraft makes before an upgrade.

You will have a private, dated copy of Kraft's database that you have checked, and the steps that put it back. Kraft also makes a copy of its own before an upgrade migrates the database, and this page says how to find it.

Before you start

  • $KRAFT_HOME is ~/.kraft unless you set it; see Environment variables. The commands find run/ as Kraft does, from $KRAFT_RUN_DIR when you set it and from $KRAFT_HOME otherwise, and config/ from $KRAFT_CONFIG_DIR or $KRAFT_HOME.
  • The commands use the sqlite3 command. Back up without sqlite3 has a Python form for a machine without it.
  • A backup needs no stop. A restore does: see Restore a backup.

Back up the database

run/orchestrator.db holds every work item, event, session and review thread. Kraft runs SQLite in WAL mode, so recent writes can sit in orchestrator.db-wal beside it. A plain copy of orchestrator.db while the server runs is not a backup: it can miss them, or not open.

  1. Use SQLite's online backup, which is safe while the server runs. The command names the file for the second you run it and prints the name. Write the name down: verify and restore need it, and it is the file to keep. A backup taken at another time never replaces an earlier one, so the copy you took before an upgrade is still there after the next one.
    -readonly opens the database you copy from read-only, so a wrong path fails instead of creating an empty database and an empty backup of it:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    backup="$HOME/kraft-backup-$(date +%Y%m%d-%H%M%S).db"
    sqlite3 -readonly "$run/orchestrator.db" ".backup '$backup'" && chmod 600 "$backup" && echo "$backup"
    
    /home/you/kraft-backup-20261002-093015.db
    

    The backup holds everything the database holds, and .backup makes it readable by other accounts under the usual umask. The chmod keeps it at 600, like the database. With no sqlite3 command, use Back up without sqlite3.
  2. Copy config/ and run/attachments/ too. A database restore brings back neither. Copy config/ whole: access.yaml holds your password hash, notify.yaml a webhook URL, and repos.yaml any env: values, so the copy stays private.
    config/ also holds addons.yaml and addons.lock, which are all a backup needs of your installed Kraft add-ons from git collections. run/addons/ needs no copy. An add-on from a directory collection comes back only while that directory still holds the same files.
    The folder is named for the second you run the command, and mkdir stops the command if the name is already there, so an earlier copy is never overwritten. Write down the name it prints: the restore needs it.
    config follows KRAFT_CONFIG_DIR, then KRAFT_TEMPLATES_DIR, when you set one. A 1.4 home that no 2.0 start has renamed keeps its configuration in templates/: put that folder in the config= line. cp -RH follows a config or attachments that is a symbolic link, so the copy holds the files, not a link that dangles without the original:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    config="${KRAFT_CONFIG_DIR:-${KRAFT_TEMPLATES_DIR:-${KRAFT_HOME:-$HOME/.kraft}/config}}"
    files="$HOME/kraft-files-$(date +%Y%m%d-%H%M%S)"
    mkdir -m 700 "$files" && cp -RH "$config" "$files/config" && cp -RH "$run/attachments" "$files/attachments" && echo "$files"
    
    /home/you/kraft-files-20261002-093015
    

    Keep your configuration in git leaves access.yaml and notify.yaml out on purpose, so a git copy alone is not a complete backup.

run/attachments/ holds the spec and plan copy of each item filed with one. After a restore, an item that has not started needs its copy: without it the item stops when its worktree is made, and the message names the missing file.

You do not need to back up run/index.db. It is the search index, and kraft admin reindex rebuilds it.

Back up without sqlite3

Python does the same as step 1, and prints the same name:

run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
backup="$HOME/kraft-backup-$(date +%Y%m%d-%H%M%S).db"
python3 -c "import sqlite3, sys; sqlite3.connect('file:' + sys.argv[1] + '?mode=ro', uri=True).backup(sqlite3.connect(sys.argv[2]))" "$run/orchestrator.db" "$backup" && chmod 600 "$backup" && echo "$backup"

Verify the backup

Open the copy and count its work items. Set backup to the file your backup step printed, so the block works in a new terminal. -readonly makes a wrong name fail instead of creating an empty file:

backup="$HOME/kraft-backup-20261002-093015.db"
sqlite3 -readonly "$backup" "PRAGMA integrity_check; SELECT count(*) FROM work_items;"

Without the sqlite3 command, set backup the same way:

backup="$HOME/kraft-backup-20261002-093015.db"
python3 -c "import sqlite3, sys; db = sqlite3.connect('file:' + sys.argv[1] + '?mode=ro', uri=True); print(db.execute('PRAGMA integrity_check').fetchone()[0]); print(db.execute('SELECT count(*) FROM work_items').fetchone()[0])" "$backup"

Both print two lines:

ok
42

You know it worked when the first line is ok and the second is the number of work items you expect.

Restore a backup

A restore replaces the database, so everything done since the backup is lost. Do the steps in order. Each block sets the shell variables it uses, so each works in a new terminal, and run follows $KRAFT_RUN_DIR, then $KRAFT_HOME. Each block that changes files in run/ refuses to run while kraft.pid is there, and says to stop Kraft first.

  1. Stop the server with kraft admin stop. If Kraft runs as a service, read the warning in Stop a service first, run kraft admin uninstall-service, then run kraft admin stop anyway: uninstall-service does not wait for the server, which can take up to 10 seconds to finish and still holds the database until it does.
    Go on only when kraft admin stop prints kraft: stopped (pid N) or kraft: no server running. If it prints kraft: pid N did not stop within 15s and exits 1, the server is still running: do not touch the database.
  2. Move the current database aside, with the files SQLite keeps beside it. Kraft runs SQLite in WAL mode, and orchestrator.db-wal holds writes that belong to the file it sits beside, so the three move together and none stays behind for the restored database to meet. The directory name carries the time, so a second restore never touches the first one's copy, and mv -n never overwrites:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    safe="$run/before-restore-$(date +%Y%m%d-%H%M%S)"
    if [ -e "$run/kraft.pid" ]; then
      echo "$run/kraft.pid exists: a server may be running. If a service runs Kraft, follow Stop a service on the Run Kraft as a service page. Otherwise run kraft admin stop, which also clears a stale pid file."
    else
      mkdir "$safe" && for f in orchestrator.db orchestrator.db-wal orchestrator.db-shm; do
        if [ -e "$run/$f" ]; then mv -n "$run/$f" "$safe"/; fi
      done && ls "$safe"
    fi
    

    The ls lists orchestrator.db, and orchestrator.db-wal and orchestrator.db-shm when the server left them. If mkdir reports an error, or ls lists nothing, stop: nothing was moved, or the database was not where this command looked. A message about kraft.pid means nothing was moved: stop Kraft, then run the block again. Run twice, this step leaves an empty before-restore-* folder: the one that holds orchestrator.db is the one that matters.
    Delete the before-restore-* folder that holds orchestrator.db once you have checked the restored database and no longer need what the old one held. Until then it is the only copy of that work.
  3. Put the backup in place as orchestrator.db, readable by you alone (0600, as Kraft makes it). Set backup to the path of the file your backup step printed, or of the copy you chose in Find the copy to restore.
    The block refuses to run while a server may be running, or while orchestrator.db, orchestrator.db-wal or orchestrator.db-shm is in run/. So a second run cannot replace a database Kraft has written since, or put the backup beside a leftover WAL file that SQLite would replay into it:
    backup="$HOME/kraft-backup-20261002-093015.db"
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    if [ -e "$run/kraft.pid" ]; then
      echo "$run/kraft.pid exists: a server may be running. If a service runs Kraft, follow Stop a service on the Run Kraft as a service page. Otherwise run kraft admin stop, which also clears a stale pid file."
    elif [ -e "$run/orchestrator.db" ] || [ -e "$run/orchestrator.db-wal" ] || [ -e "$run/orchestrator.db-shm" ]; then
      echo "a database file is already in $run: move the old database aside first, unless the backup is already in place"
    elif [ ! -f "$backup" ]; then
      echo "$backup is not a file: set backup to the file your backup step printed"
    else
      cp "$backup" "$run/orchestrator.db" && chmod 600 "$run/orchestrator.db" && ls -l "$run/orchestrator.db"
    fi
    

    When it works it prints one line for the restored file, which starts with -rw-------. If a message appears, nothing was copied: act on it and run the block again. If cp prints an error, do not start Kraft: it would create an empty database and show an empty board. Move the half-written copy aside as step 2 does, then run this block again.
  4. Put config/ and run/attachments/ back, if you lost them or they are wrong. Do it while the server is still stopped from step 1. A database restore does not touch them.
    The block overwrites each file of the same name in config/ with the copy from step 2 of Back up the database, so it first copies the config/ you have to a dated folder beside it and prints the name. It stops, restoring nothing, if that copy cannot be made. Set files to the folder that step printed. config follows $KRAFT_CONFIG_DIR, then $KRAFT_TEMPLATES_DIR:
    files="$HOME/kraft-files-20261002-093015"
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    config="${KRAFT_CONFIG_DIR:-${KRAFT_TEMPLATES_DIR:-${KRAFT_HOME:-$HOME/.kraft}/config}}"
    config="${config%/}"
    saved="$config.before-restore-$(date +%Y%m%d-%H%M%S)"
    if [ -e "$run/kraft.pid" ]; then
      echo "$run/kraft.pid exists: a server may be running. If a service runs Kraft, follow Stop a service on the Run Kraft as a service page. Otherwise run kraft admin stop, which also clears a stale pid file."
    elif [ ! -d "$files/config" ]; then
      echo "$files/config is not a folder: set files to the folder your files copy printed"
    elif [ -e "$saved" ]; then
      echo "$saved is already there: run this again"
    elif [ -d "$config" ] && ! cp -RH "$config" "$saved"; then
      echo "could not copy $config to $saved: nothing was restored. Delete $saved if it is there, fix the cause, and run this again"
    else
      if [ -d "$saved" ]; then echo "the config you had is kept at $saved"; fi &&
        mkdir -p "$config" "$run/attachments" &&
        cp -R "$files/config/." "$config/" &&
        if [ -d "$files/attachments" ]; then cp -R "$files/attachments/." "$run/attachments/"; fi &&
        ls "$config"
    fi
    

    It names the folder that keeps the config you had, then lists the files that are back. If cp prints an error after that name, the restore stopped partway: the named folder still holds everything you had. Any other message means nothing was restored: act on it and run the block again.
    Each run makes a new dated folder, so the oldest one holds what you had before the first run. Delete it once Kraft runs on the restored config and you no longer need what the old files held. Until then it is the only copy of them.
    attachments/ is only added to: cp -R replaces a file of the same name and removes none, so an attachment you added since the backup stays.
  5. Start the Kraft release that wrote the backup, with kraft admin start --detach (or kraft admin install-service if you removed the service in step 1). To go back a release first, see Pin or roll back a version.

kraft admin start --detach prints the address and the server's process id:

kraft: http://127.0.0.1:8765 (pid 48213, detached - kraft admin stop)

You know it worked when kraft view list --all shows the items the backup held. run/index.db needs no restore: the server rescans repo documents when it starts, and kraft admin reindex rescans them on demand.

run/addons/ needs none either. When the restored config/ holds addons.lock, the server fetches each installed add-on again at its locked commit after it starts, in the background, and loads it once its files match the locked digest. The same happens when you copy config/ to a new machine. The machine needs git access to each collection's URL. kraft admin doctor names an add-on that is not back on its add-ons row. See Restoring a store.

If the restore went wrong

If the board is empty or shows the wrong items, put the old database back. The copy in the before-restore-* folder is untouched.

  1. Stop the server, as in step 1.
  2. Move the restored files into a new directory, so nothing is overwritten:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    bad="$run/restored-$(date +%Y%m%d-%H%M%S)"
    if [ -e "$run/kraft.pid" ]; then
      echo "$run/kraft.pid exists: a server may be running. If a service runs Kraft, follow Stop a service on the Run Kraft as a service page. Otherwise run kraft admin stop, which also clears a stale pid file."
    else
      mkdir "$bad" && for f in orchestrator.db orchestrator.db-wal orchestrator.db-shm; do
        if [ -e "$run/$f" ]; then mv -n "$run/$f" "$bad"/; fi
      done && ls "$bad"
    fi
    
  3. Move the old files back. This block sets its variables itself, so it is safe in a new terminal. The first line lists the orchestrator.db each before-restore-* folder holds. Put the name of the right folder, which step 2 of the restore made, in place of before-restore-20261002-093015. The block moves nothing while a server may be running or a database file is in run/:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    ls -d "$run"/before-restore-*/orchestrator.db
    safe="$run/before-restore-20261002-093015"
    if [ -e "$run/kraft.pid" ]; then
      echo "$run/kraft.pid exists: a server may be running. If a service runs Kraft, follow Stop a service on the Run Kraft as a service page. Otherwise run kraft admin stop, which also clears a stale pid file."
    elif [ -e "$run/orchestrator.db" ] || [ -e "$run/orchestrator.db-wal" ] || [ -e "$run/orchestrator.db-shm" ]; then
      echo "a database file is still in $run: move the restored files aside first"
    elif [ ! -d "$safe" ]; then
      echo "$safe is not a directory: check the name against the ls above"
    else
      for f in orchestrator.db orchestrator.db-wal orchestrator.db-shm; do
        if [ -e "$safe/$f" ]; then mv -n "$safe/$f" "$run"/; fi
      done && ls "$run"/orchestrator.db*
    fi
    

    A message means nothing was moved: act on it. Otherwise the last ls lists orchestrator.db back in run/. If it lists nothing, the folder was an empty one: pick the one the first line listed and run the block again.
  4. Start Kraft as in step 5 of the restore.

The copy Kraft makes before an upgrade

When a newer Kraft starts on an older database, it migrates the database forward, and the older version cannot open the result. So before it migrates, Kraft copies the database to run/orchestrator.db.pre-v<N>-<time>, where <N> is the schema version it is about to migrate to and <time> is when it made the copy. Two copies in the same second get -2, -3 and so on after the time. run/logs/server.log names the file:

database schema v44 -> v52: backed up the old database to /home/you/.kraft/run/orchestrator.db.pre-v52-20261002-093015

The copy is consistent even while the WAL holds recent writes, and only you can read it (0600). Each migration makes a new copy and never overwrites an older one. If you upgrade, roll back and upgrade again, the newest orchestrator.db.pre-v52-* is the one that holds what you did in between.

The copy needs about as much free space as the database, beside it in run/. If Kraft cannot write it, it does not migrate: the server does not start, the database is left unchanged, and the error names the copy and the space it needs. Free the space and start again.

The copy loses everything done since the upgrade, and it is no substitute for your own backup: make one before every upgrade. Once you are sure you will stay on the new version, delete the copies. Kraft never deletes them itself.

Find the copy to restore

The v<N> in the name is the schema Kraft migrated to, not the one the copy holds. After more than one upgrade, the newest copy can hold a schema the release you go back to refuses. Restore the newest copy whose own schema is the one that release runs, and 1.4's schema is 44.

  1. Print each copy's schema, then its name, one line per copy. This only reads the copies: -readonly makes a missing file an error instead of an empty file named like a copy. Names sort by schema, then by time:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    for f in "$run"/orchestrator.db.pre-v*; do echo "$(sqlite3 -readonly "$f" 'PRAGMA user_version') $f"; done
    

    Without the sqlite3 command, Python prints the same:
    run="${KRAFT_RUN_DIR:-${KRAFT_HOME:-$HOME/.kraft}/run}"
    python3 -c 'import glob, sqlite3, sys; [print(sqlite3.connect(f"file:{f}?mode=ro", uri=True).execute("PRAGMA user_version").fetchone()[0], f) for f in sorted(glob.glob(sys.argv[1] + "/orchestrator.db.pre-v*"))]' "$run"
    
    44 /home/you/.kraft/run/orchestrator.db.pre-v52-20261002-093015
    

    An error naming orchestrator.db.pre-v*, or no lines, means Kraft made no copy.
  2. Pick the newest line that starts with the schema you are going back to, such as 44.
  3. Restore that file as in Restore a backup, with backup set to its path.

For 1.4's database, the first release you ran after 1.4 decides which copy holds it, and a 1.5.0 release candidate may have made none: see Upgrading from a 1.5.0 release candidate.

Next steps

Copyright © 2026