Documentation

Configuration

This page is the guide to .sift.toml, the per-repository config file. Flags configure a single run; profiles and targets make settings reusable and committable so sift dump does the right thing without flags — suitable for CI and automated dumping. For the full flag list see Flag reference.

Quick start: [sift] defaults

Put repository-wide defaults in .sift.toml (project-local, committed) so any clone can run sift dump with no flags:

[sift]
style = "xml"
budget = 40000
compress_mode = "full"
output = "codebase.md"
smart_filter = true
extensions = ["go", "md", "ts"]
ignore = ["vendor/**", "dist/**"]
prompt = "Review for correctness."
# copy_on_generate = true  # also copy each generated document to the clipboard
# prompt_file = "prompts/review.md"   # file path instead of inline prompt

Every field is optional. Anything you omit keeps its built-in default, so start with only the two or three settings you actually care about.

Flags still win: sift dump --budget 80000 overrides budget = 40000.

Where the file is looked up

The local file is resolved against the directory you are scanning, not your current working directory:

  • sift dump / sift pick in a repo → ./.sift.toml
  • sift dump /path/to/repo → /path/to/repo/.sift.toml

This means CI and tooling that scan a checkout from elsewhere still pick up the committed config. A relative --dir is resolved to an absolute path before the lookup, so sift dump . and sift dump /abs/path behave the same.

Key names

TOML keys are snake_case and do not always match the flag spelling:

TOML key Flag Notes
compress_mode --mode Values: full, signatures, skip
extensions --ext Comma-separated on the CLI, a TOML array in the file
ignore --ignore Globs in the file, comma-separated on the CLI
smart_filter --smart Name differs from the flag
smart_max_tokens --smart-max-tokens Per-file token ceiling
copy_on_generate --clipboard Name differs from the flag
tokenize_model --tokenize-model e.g. cl100k_base
ui_theme_file --ui-theme-file Picker themes only

Inverted booleans

Several settings are phrased as “ignore/skip X” in TOML but as “include X” on the command line, because the flags default to filtering. This is the most common source of confusion in the file:

TOML key Default Set to false to Flag (mirror image)
ignore_hidden true include hidden files --hidden=false
ignore_git true include Git-ignored files --git=false
secret_scan true disable secret redaction --secrets=false
smart_filter false disable the smart filter --smart=false

So ignore_hidden = false means “do not ignore hidden files” — the opposite of how it reads at a glance.

ignore_git / --git controls .gitignore handling, and the .git directory is always skipped. Note the flag’s help text says “Ignore .git directories”, which is misleading — the setting actually turns off .gitignore rule matching.

One exception: the interactive picker always walks hidden and Git-ignored files so you can opt into them with its visibility toggles, regardless of these settings. They apply to dump and select.

force_secrets = true includes detected credentials verbatim instead of redacting them. Only use this on trusted input.

Lists

extensions and ignore are TOML arrays:

extensions = ["go", "md", "ts"]
ignore = ["vendor/**", "dist/**", "**/*.min.js"]

ignore entries are glob patterns matched against repository-relative paths. .gitignore is honored independently; these patterns are additive.

Keep generated output out of the scan

The document sift writes lands inside the repository by default, so on a second run it can be picked up and fed back into the next document. Add the output names to ignore (and to .gitignore) if that matters to you:

[sift]
output = "codebase.md"
ignore = ["codebase.md", "docs-context.md"]

Or point output outside the tree, or use - for stdout.

Profiles

Profiles are named presets you select with --profile (or via a target):

[profiles.review]
style = "markdown"
budget = 60000
compress_mode = "signatures"
prompt = "Review this codebase for correctness and security issues."
extensions = ["go", "md"]

[profiles.claude]
style = "xml"
budget = 60000

[profiles.rust-strict]
extends = ["claude"]           # inherit, then override
extensions = ["rs"]
compress_mode = "signatures"
prompt_file = "prompts/rust-review.md"
output = "rust-context.md"
sift dump . --profile review
sift dump . --profile rust-strict

extends composes profiles: ancestors are overlaid in order, then the profile’s own fields win. Cycles are an error.

You can make one profile the default so plain sift dump uses it:

default_profile = "claude"

A default profile still overrides [sift], because profiles sit later in the chain. If you set both, the profile’s values win.

::: warning Top-level keys must precede the first table header default_profile (and any other bare key) must appear before the first [table] header. TOML assigns a bare key to whatever table is currently open, so this silently does nothing:

[profiles.review]
prompt = "Review this."

default_profile = "review"   # parsed as profiles.review.default_profile!

There is no error — the profile just never activates, and [sift] values are used as if you had not configured a default. Put top-level keys at the top of the file.

Targets: multiple artifacts from one file

Each [[targets]] is one dump artifact. Select one with --target:

[[targets]]
name = "full"
profile = "claude"             # optional; inherits that profile first
output = "codebase.md"

[[targets]]
name = "sig"
profile = "rust-strict"
output = "codebase.sig.xml"
style = "xml"

[[targets]]
name = "docs"
extensions = ["md"]
prompt = "Summarize the architecture for a new teammate."
output = "docs-context.md"
budget = 12000
sift dump --target full        # codebase.md
sift dump --target sig         # codebase.sig.xml
sift dump --target docs        # docs-context.md (md-only, short budget)

Target fields (style, budget, compress_mode, output, prompt/prompt_file, extensions, ignore, scoring) override the resolved profile and [sift] defaults. Flags override everything.

Targets are the cleanest way to keep several committed artifacts in one repo — one --target per artifact, each writing its own file.

Prompts

Inline prompt or file-backed prompt_file on [sift], any profile, or any target:

[profiles.review]
prompt_file = "prompts/review.md"

[[targets]]
name = "api"
prompt_file = "prompts/api.md"

A [prompts.NAME] library is also parsed:

[prompts.review]
text = "Review for correctness and security."

[prompts.docs]
file = "prompts/docs.md"

These named entries are only reachable from the Go API today (config.WithPromptRef); there is no CLI flag for them yet. Use inline prompt or prompt_file on a profile or target until that lands. A [prompts.*] entry with both text and file set is an error.

Automation hints

Declarative hints for watch / hooks (opt-in; nothing runs from TOML alone):

[automation]
watch_interval = "500ms"
git_hook = "post-commit"       # `sift hook install` reads this
hook_targets = ["full", "sig"]

Precedence

Built-in defaults (Config.New())
  → global ~/.config/sift/config.toml
  → local .sift.toml [sift] defaults
  → default_profile (if set)
  → selected profile (+ extends expansion)
  → selected target overrides
  → explicit --flags (pflag.Changed guard)

Local .sift.toml overlays global; sift dump /path resolves .sift.toml relative to the scanned root, not just cwd. Within a layer only the fields you actually set are applied, so a local file can override a single global key without restating the rest.

Legacy

Bare [profiles.NAME] without [sift]/[[targets]] continues to work, and so does default_profile in either config file.

extensions and ignore as TOML arrays (e.g. extensions = ["go", "md"]) are canonical; comma strings are a CLI concern (--ext go,md), not TOML.

What applies to which command

.sift.toml is read by every subcommand, but not every key means the same thing everywhere. The differences are by design:

Key dump select pick (TUI)
extensions yes yes yes (filters the walk)
ignore yes yes yes (filters the walk)
ignore_git, ignore_hidden yes yes no — always walked
budget yes yes yes (shown in the footer)
prompt yes yes yes (prompt bar)
style, scoring yes n/a yes (preview styling)
window_title n/a n/a yes
compress_mode yes no per-file, interactive
skip_roles no yes yes (smart-select)

Two entries deserve explanation:

  • compress_mode only affects dump. sift select derives a mode per file from the relevance ranker (git recency, churn, centrality, role, and compression yield). Forcing it uniform would defeat the command, so your [sift] compress_mode is deliberately ignored there. Tune [scoring] instead.
  • skip_roles only affects select and the picker. dump has no role classifier, so to skip tests in a dump you need ignore globs such as "*_test.go". Conversely, select and the picker already skip test/fixture/mock roles automatically unless the task prompt is about tests.

smart_filter is forced on for select and pick regardless of the setting, since those workflows would otherwise surface generated and lock files.

ignore_git and ignore_hidden are the reverse exception: the picker always walks hidden and Git-ignored files so its visibility toggles have something to toggle, so those two keys only affect dump and select.

Tuning selection ([scoring])

Any [sift], profile, or target may carry a scoring table. Unset keys keep built-in defaults; retention overrides per-role retention under budget (config priority over language default):

[profiles.min]
scoring = { retention = { docs = 0.0 } }        # don't guarantee docs under budget

[profiles.lean]
[profiles.lean.scoring]
recency_weight = 0.40
churn_weight   = 0.20
centrality_weight = 0.20
role_weight    = 1.0
full_band      = 0.55
skip_band      = 0.20
base_min       = 0.01
base_max       = 1.0
relevance_weight = 0.20
preference_bonus = 0.05
signature_bonus  = 0.05
skip_multiplier  = 0.35
sig_quality_min  = 0.35
sig_quality_max  = 0.85
test_task_boost  = 0.20
area_diminishing = 0.65
area_budget_share = 0.35
area_roots       = { "internal/ui" = "frontend", "pkg/api" = "public-api" }
skip_roles       = ["test", "fixture", "mock", "generated", "vendor"]
retention        = { entrypoint = 0.30, docs = 0.22, config = 0.18, api = 0.12 }

Top-level [scoring] is also honored. The default skip_roles are tests, fixtures, mocks, generated files, and vendored code. With the default policy, test files become eligible and receive test_task_boost when the task prompt mentions tests, bugs, regressions, coverage, or fixing an issue. An explicit skip_roles list replaces the defaults and always takes precedence. Automatic selection first reserves strong prompt matches and an affordable representative from each architectural area, then enforces a soft per-area budget ceiling and applies diminishing utility to later files in the same area. Role-based retention is limited to two reserved files per area, while recent implementation files get at most one reservation per area.

Areas default to the first two path components (internal/app, cmd/sift); root files and files directly under a top-level directory share their respective areas. area_diminishing controls the multiplier for the second file and repeated later files (default 0.65, with a floor of 0.20). area_budget_share sets the per-area budget ceiling (default 0.35; the effective share is raised when there are few areas). area_roots maps directory prefixes to custom area names; the longest matching prefix wins.

Settings reference

Output

Flag Default TOML key Purpose
--output codebase.md output Output file; - writes to stdout
--style markdown style plain, markdown, json, or xml
--mode full compress_mode full content or signatures outline
--clipboard off — Copy the rendered output to the system clipboard
--copy-on-generate off copy_on_generate Also copy generated documents while writing them to the output file
--target — — Target from .sift.toml (--target NAME)
--profile — — Profile from config (--profile NAME)

Context and tokens

Flag Default Purpose
--budget 0 Maximum output tokens; 0 is unlimited
--tokenize-model cl100k_base BPE encoding used for counting
--prompt, -p — Task directive prepended to the output

Filtering

Flag Default TOML key Purpose
--ext — extensions Only include these extensions (comma-separated)
--ignore — ignore Extra gitignore-style patterns (comma-separated)
--max-size 0 — Max file size in MB; 0 is unlimited
--hidden true ignore_hidden Ignore hidden files; set false to include them
--git true ignore_git Honor .gitignore; set false to include ignored files (.git is always skipped)
--binary off — Include binary files (skipped by default)
--smart off smart_filter Skip generated/lock/minified/oversized files
--smart-max-tokens 15000 smart_max_tokens Per-file token ceiling for the smart filter

Safety

Flag Default TOML key Purpose
--secrets true secret_scan Scan and redact detected credentials
--force-secrets off force_secrets Include secrets instead of redacting

Terminal display

Flag Default Purpose
--highlight true Terminal syntax highlighting
--theme auto Highlight palette: auto, none, dark, light
--ui-theme-file — TOML file containing custom interactive picker themes
--no-color off Disable color output
--window-title — Set the picker’s terminal title
--no-window-title off Disable terminal title updates
--no-nerd-fonts off Plain ASCII glyphs in the picker

Scan behavior

Flag Default Purpose
--dir . The root directory to scan
--concurrent true Concurrent file processing
--workers CPU count Number of concurrent workers
--timeout — Maximum execution time (e.g. 30s, 5m)
--show-skipped off List excluded files and reasons
--progress off Show progress information

Diagnostics

Flag Default Purpose
--verbose off Debug-level logging
--quiet off Only warn/error logging
--log-level — Set the logging level explicitly

Interactive theme vs. syntax theme

Two related but separate settings:

  • --theme controls terminal syntax highlighting of rendered output (auto, none, dark, light). Color output automatically degrades for truecolor, ANSI256, ANSI16, and no-color terminals.
  • The picker’s interactive color palette (chosen with t in the TUI) is stored separately under the user’s application configuration directory and reloaded on the next picker session. Inside the picker, the active palette also drives preview syntax highlighting; --theme still governs non-interactive output. The built-in Terminal (Emulator) palette uses the terminal’s configured ANSI colors rather than fixed RGB values.

Custom picker themes

Pass a TOML file with --ui-theme-file (or set ui_theme_file in .sift.toml):

[[theme]]
id = "midnight"
name = "Midnight"
extends = "catppuccin-mocha"
border = "#112233"
title = "#abcdef"

[[theme]]
id = "midnight-soft"
name = "Midnight Soft"
extends = "midnight"
selected = "#89b4fa"

Each theme requires a stable id and display name. Themes may inherit from a built-in theme or an earlier definition in the same file. The file is loaded only by the interactive picker; it does not change the --theme setting used by non-interactive output. Theme IDs are persisted across sessions, while older saved display names remain supported.

Where state lives

Per-project dump state (used by delta) and interactive preferences are kept outside the scanned repository — by default under the application configuration directory (e.g. ~/.config/sift/). Nothing is written into your project, so the context document and repo stay clean.