Denpie

Daily tip cards with SM-2 review. Rust/Axum backend. Any OpenAI-compatible LLM.

Run it now

just shell       # or nix-shell  (~30s first time)
just setup       # verify toolchain
just db-up       # start local PostgreSQL
just dev         # backend + Astro watchers. Open http://localhost:4321/

No Nix? Need Rust 1.95.0, protoc, bun, and Docker Compose (or PostgreSQL 19 beta 2 + psql). Pin is in rust-toolchain.toml.

First boot (~2 min)

cargo run

Set DATABASE_URL first (the checked-in .env.example matches just db-up). Denpie applies embedded SQLx migrations, builds the frontend if needed, and prints a one-time admin_token.

1. Open http://127.0.0.1:3017/
2. Create the first admin user with the printed token
3. Create an API key (UI) or bootstrap_api_key with that token
4. Set LLM model, API key, base URL, prompt (Settings UI or update_settings)
5. Put the key in ApiRequest.auth for every POST /api call

Backend-only hacking: DENPIE_SKIP_FRONTEND_BUILD=1.

What it does

Core

1. SM-2 scheduling — grades, due windows, card types
2. Daily topic cards — per-topic refresh windows; Force Daily Refresh loads fresh cards
3. One public API — POST /api (protobuf); / is the browser app
4. Multi-user isolation — topics, cards, reviews, settings, keys
5. Tipcard images — browser compresses; server rejects >10 MB decoded, recompresses >800 KB

Also

• Manual cards (no LLM) and custom_tip cards (no review state)
• Topic icons (Iconify + HSL accent; fallback lucide:tag)
• Pinning + max_active_cards cap
• Token spend counters (daily / monthly / lifetime)
• Optional GitHub self-updates via systemd (off by default)

Screenshots

Grounding	Transmission	Fullscreen Card

Transmission keeps pinned cards in a separate top section. Beneath it, topic picks show up to three cards per topic and nine cards total, reducing the per-topic allowance evenly when the total would exceed nine. All remaining due cards continue in an "Other cards" section below the picks. The grid switch opens a quick column picker; the saved maximum remains constrained by the current page width.

Dev commands

just quick            # fmt check + compile (default while editing)
just api-check        # additive v1 wire/operation/result contract
just test-one <filter>  # targeted tests
just verify           # one full gate: fmt + clippy + tests
just docs-check       # generated API reference + four executable client examples
just lab list         # opt-in research benches (not CI)
just lab run images --dry-run  # print the image bake-off plan (no network)
just lab run cards --dry-run   # print repeatable-card fixture states (no network)
just lab compare <old.json> <new.json>  # compare image or prompt scorecards
just lab-check        # deterministic offline lab contract
just lab-cards-dev    # production card fixtures with HMR on isolated :3027
just lab-cards-ui     # production FlowCard fixtures on isolated :3027
just lab-review <baseline> <candidate>  # blinded A/B review workbench on :3027
just agent-server     # isolated :3027 runtime, test login, smoke
just playwright-install # install local Playwright + Chromium (once per clone)
just playwright       # headless Chromium UI smoke against isolated :3027
just frontend-astro-test  # Astro catalog + Bun tests
just ui-check         # Astro build + agent oneshot smoke
just ci               # verify + Astro tests and release build

just lab is the opt-in research runner (see docs/lab.md). It is not part of just test, just verify, or just ci; live just lab run images and just lab run prompts use the network. run cards builds its gallery locally. just lab-cards-ui mounts the checked-in states in the production review-slot card at /lab-cards on :3027. Its actions are local simulations and cannot mutate server images. just lab-review loads two run artifacts into a blinded, exportable A/B review. New UI work: docs/frontend-astro.md.

RUST_LOG=denpie=debug just backend

Grounding/image strategies log stage progress at info. LLM transport detail is at debug. Chat/vision calls use a dedicated 300s HTTP client (the shared client stays at 60s). Parse or body-read failures log the error source chain plus a bounded head+tail body snippet at warn; success bodies are not logged at info.

Docs map

Need	Open
Recommended API v1	docs/api-v1.md
Complete v1 operation table	docs/api-v1-reference.md
API examples (curl, Python, TypeScript, Rust)	examples/api/README.md
API schema bundle	api/schema/v1/README.md
Rules for adding/changing API operations	docs/api-development-rules.md
API compatibility and changelog	docs/api-compatibility.md, docs/api-changelog.md
POST /api compatibility reference	docs/protobuf-api.md
Agent ops cheat sheet	docs/agent-server-guide.md
Where new code goes	docs/feature-integration.md
Browser UI	docs/frontend-astro.md
Opt-in research lab	docs/lab.md

Config

Global — settings.yaml (generated, do not commit)

Key	Default	What
admin_token	auto	First-user setup token
autoupdate_enabled	false	GitHub self-updates
autoupdate_repo	slopfire/denpie	owner/repo or URL
autoupdate_branch	master	Branch to watch
autoupdate_check_interval_secs	3600 (min 60)	Poll interval
autoupdate_command	empty	Non-systemd update command
autoupdate_last_seen_sha	empty	Last seen remote SHA

Per-user (PostgreSQL)

Page	Owns
Settings	Default LLM, endpoints, credentials, prompt, reasoning/compression, appearance, schedule, max_active_cards
Grounding	Grounding-agent model/reasoning, fact grounding, Tavily or Firecrawl, link scraper, image retrieval

Empty grounding-agent fields inherit default LLM settings.

Generated-card prompts include a single compact list of existing titles: up to 24 recent unlabeled titles, plus any known / hard / skip titles from the last 80 cards. Each title appears once, with an inline review label when there is feedback. Card bodies are not sent. The default prompt template is batch-agnostic (Write useful daily tip cards about {topic}) so agentic/RAG wrappers do not stack a second “write one 180–260 word tip” brief. An empty topic prompt uses the user template; an empty user template uses that built-in default. Settings and the topic editor can reset a template by pasting the current prompt (built-in default, or the global settings template on a topic) and run Enhance, which reads that title history and fills a suggested prompt plus optional grounding changes. Save still applies them.

Image modes: No Images · Local Image Pool · Bing Images (HTML) · Bing Images (Playwright) · DDGS + Open Graph.

The Tavily/Firecrawl web provider is used for factual grounding. Automatic image retrieval is keyless: Bing HTML is the direct built-in path. The explicit Playwright option uses the repository's local Node/Chromium installation; DDGS + Open Graph uses python3 with the optional ddgs package.

Link scraping is separate. The main option is Scrapling (local CLI): when installed it converts linked pages to clean, AI-targeted Markdown. Alternatives: Firecrawl cloud scrape (/v2/scrape, including remote PDFs) or legacy direct HTTP. Install with pip install "scrapling[fetchers,shell]". The Firecrawl search base URL can target the hosted API or a compatible self-hosted deployment.

Local Image Pool uploads accept PNG, JPEG, WebP, and GIF images up to 10 MB decoded. Denpie recompresses larger uploads before sending them to the configured vision model for automatic naming, description, and tags. An empty Vision Model setting inherits the default LLM model; annotation failures retain the user-entered fallback name. The Settings vision test and annotation requests disable reasoning and leave a 1024-token completion budget so thinking models (for example MiniMax-M3) do not spend a tiny max_tokens cap on hidden thinking and return empty content.

Generated cards request an image only when the model marks a visual as materially useful (for example a diagram, physical identification, UI screenshot, or comparison). The decision and specific query are stored with the card, including pending agentic-backlog cards; manual and custom cards do not trigger automatic retrieval. The three remote modes discover source URLs through Bing HTML, rendered Bing HTML, or DDGS text results and page Open Graph metadata. Every candidate is downloaded through the shared DNS-pinned, redirect-validating image path. Generated cards enqueue durable image jobs: provider latency never blocks card promotion, failed jobs retry with leases, and storage completion is idempotent after worker restarts.

Legacy stored values programmatic, agentic, and web_search resolve to bing_html. Disable the optional process modes with DENPIE_DISABLE_BING_PLAYWRIGHT=1 or DENPIE_DISABLE_DDGS=1; override their executables with DENPIE_PLAYWRIGHT_BIN and DENPIE_DDGS_BIN.

Image enrichment runs for active and pending generated cards. Pending cards and their attached-image previews are available in Archive without entering the active review flow; the image also appears on the review card after promotion. Attachment logs include card status and only image metadata, never the image byte payload.

Card image append endpoint

POST /app/tipcard-images/append is session-authenticated and accepts { "card_id": 1, "image_data": [], "pool_image_ids": [], "urls": [] }. It appends up to four card images total rather than replacing existing attachments. Data URLs and remote downloads are limited to 10 MB decoded each; the JSON request limit is 56 MB. Pool images are copied into card-owned storage. URL downloads allow only credential-free HTTP(S) targets, reject private and local network addresses, validate every redirect, and cap redirects and response bytes.

• UI shows only providers for the selected mode.
• Providers start disabled — enable at least one.
• Denpie tries enabled providers in order until one returns a valid image.

Topic cards link to their queued pending cards and reviewed SM-2 scheduled cards in the Archive.

Self-updates

1. Set autoupdate_enabled
2. Helper rebuilds frontend + backend, installs, records SHA, restarts denpie.service
3. Status: /admin/autoupdate/status
4. Timeouts: network/restart 120s · build 1800s · install 300s

Env vars

Variable	Default
DATABASE_URL	required
DENPIE_DB_SCHEMA	public
DENPIE_BIND_ADDR	127.0.0.1:3017
DENPIE_RP_ORIGIN	http://localhost:3017
DENPIE_RP_ID	from DENPIE_RP_ORIGIN
DENPIE_RP_EXTRA_ORIGINS	none
DENPIE_PROD	off (on for https)
DENPIE_DATA_DIR	current directory
DENPIE_FRONTEND_DIST	./frontend-astro/dist
DENPIE_STATIC_DIR	./static
DENPIE_IMAGE_DIR	$DENPIE_DATA_DIR/tipcard-images
DENPIE_DISABLE_BACKGROUND_JOBS	unset (jobs run)

Deploy

systemd (~1 min)

DATABASE_URL='postgres://denpie:secret@db.example.com/denpie' ./install.sh

On first install, DATABASE_URL is required and is stored root-only in /etc/default/denpie. The installer deploys the binary/frontend/static assets, creates the denpie user, enables denpie-autoupdate.timer, and restarts.

BIND_ADDR=127.0.0.1:3010 RP_ID=example.com RP_ORIGIN=https://example.com ./install.sh

Admin token after first start:

sudo journalctl -u denpie -n 100 --no-pager

Docker

docker build -t denpie .
docker run -d --name denpie --network host \
  -e DATABASE_URL=postgres://denpie:secret@127.0.0.1:5432/denpie \
  -e DENPIE_RP_ORIGIN=https://denpie.example.com \
  -e DENPIE_RP_ID=denpie.example.com \
  -v denpie-data:/var/lib/denpie \
  denpie

• Host ownership: DENPIE_UID / DENPIE_GID
• Reverse proxy: DENPIE_BIND_ADDR=0.0.0.0:3017

Existing SQLite data

Keep the old file as a backup, start an empty PostgreSQL database, then run the one-shot importer:

cp denpie.db denpie.db.backup
just db-up
scripts/migrate-sqlite-to-postgres.py --sqlite denpie.db

The importer opens SQLite read-only, refuses a non-empty PostgreSQL target, preserves IDs, repairs historical orphan owners/topics with labeled placeholders, resets PostgreSQL sequences, and verifies every table count before committing. Use --database-url and --schema for a non-default target.

Set DENPIE_DISABLE_BACKGROUND_JOBS=1 while validating a migrated database. This keeps the HTTP service available but does not start automatic update checks, daily refreshes, or image-enrichment workers. Remove the variable after the cutover is accepted.

DockerHub CI

Secrets: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN, optional DOCKERHUB_REPOSITORY.
Tags: branch, Git tag, sha-<commit>, latest.

Layout

src/
  main.rs app.rs auth.rs error.rs types.rs
  api/          # protobuf handlers (thin)
  services/     # orchestration
  domain/       # pure rules — no SQL, no YAML
  db/repositories/
  dashboard/    # browser handlers
  llm/ scheduling/ autoupdate/ config/ tests/
proto/denpie.proto
migrations/    # embedded SQLx PostgreSQL migrations
schema.sql     # canonical fresh PostgreSQL schema
frontend-astro/ # Astro + React islands (served UI)
settings.yaml   # local only — do not commit

Tables

Table	Holds
api_keys	SHA-256 hashed client keys
users	Profiles, roles, avatars
topics	Type, prompt, icon, color, daily overrides
tipcards	Content, title, pin state
review_states	SM-2 state, learning feedback, active/pending deck status, repeats, next review
tipcard_images	Attachment metadata
card_image_jobs	Durable automatic-image enrichment leases and retry state
user_documents / document_topics	Grounding sources + topic links
image_pool	Local image pool entries
llm_token_usage	Per-call token totals
user_settings	LLM / UI / schedule
daily_refresh_runs	Processed topic windows
passkeys	WebAuthn credentials

Tests

just test

Integration tests use real servers on ephemeral ports, isolated PostgreSQL schemas, and isolated temp settings. Start the local database with just db-up; just ci also runs fmt, clippy, Astro tests, and an Astro release build.

Stack

Layer	Tech
Language	Rust 2024
Web	Axum
DB	PostgreSQL + SQLx
Runtime	Tokio
LLM	async-openai + dedicated 300s reqwest client
Wire format	Protobuf (prost)
Frontend	Astro + React + Tailwind v4 (shadcn CLI) in frontend-astro/
Public API	POST /api

License

MIT — LICENSE.