152 خطوط
11 KiB
Markdown
152 خطوط
11 KiB
Markdown
# Native Altium schematic rendering service
|
|
|
|
A working REST upload → durable worker → native SchDoc parser → versioned geometry/SVG → fullscreen viewer pipeline. The viewer consumes uploaded schematic records, including placed component symbol primitives; no sample circuit or substitute component library is used.
|
|
|
|
**Fidelity status: partial compatibility, not pixel-perfect or Altium-certified.** Parsing and rasterization have been exercised with real binary schematics. A trusted export from Altium is required before any claim of visual equivalence. Unsupported objects and text/font limitations are recorded in every result. See [compatibility](docs/COMPATIBILITY.md) and [validation](docs/VALIDATION.md).
|
|
|
|
## Run with Docker
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Replace API_KEY and POSTGRES_PASSWORD with independent random values.
|
|
# openssl rand -hex 32
|
|
# A private local .env may already exist if this workspace was initialized.
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Open `http://127.0.0.1:8000` for the optional upload form, or use the REST API. The actual viewer is `/view/{id}` and contains only the drawing after it loads. Local deployment binds the API explicitly to loopback. The API and worker run as UID 10001 with read-only root filesystems, dropped capabilities, memory limits, and private persistent storage. PostgreSQL has no published port.
|
|
|
|
For a production domain, set `DOMAIN`, `APP_ENV=production` and `PUBLIC_BASE_URL=https://your-domain.example` in `.env`, configure DNS to the server, and run:
|
|
|
|
```bash
|
|
docker compose -f compose.yaml -f compose.production.yaml up -d --build
|
|
```
|
|
|
|
The production overlay adds Caddy for HTTPS. Production startup rejects short API keys and non-HTTPS public URLs. Change the Caddy request-size limit when changing the upload limit. Use a URL-safe database password (random hex is suitable) because it is interpolated into a database URL. Back up the PostgreSQL and schematic volumes together. The initial schema is created under an advisory lock; subsequent schema changes require a reviewed migration, not automatic destructive schema synchronization.
|
|
|
|
To scale CPU-bound parsing:
|
|
|
|
```bash
|
|
docker compose up -d --scale worker=4
|
|
```
|
|
|
|
PostgreSQL row locks and compare-and-swap claims prevent duplicate active claims; leases recover crashes, and attempt-specific artifacts prevent a stale worker overwriting a newer result. PostgreSQL is also the durable queue, avoiding a database/Redis enqueue consistency gap. SQLite is available for local development only.
|
|
|
|
## Run locally
|
|
|
|
Python 3.12 and Node 24 (tests only) are used. Cairo is required only by the optional raster validation tool.
|
|
|
|
```bash
|
|
python3 -m venv .venv
|
|
.venv/bin/pip install -r backend/requirements-dev.txt
|
|
./scripts/dev.sh
|
|
```
|
|
|
|
Without a `DATABASE_URL`, the app uses SQLite in `data/`. `scripts/dev.sh` starts an API and worker, sourcing `.env` if present. Upload authentication is disabled only when no `API_KEY` is configured in development. Do not expose that mode publicly.
|
|
|
|
## API example
|
|
|
|
```bash
|
|
curl -X POST http://127.0.0.1:8000/api/v1/schematics/render \
|
|
-H "X-API-Key: $API_KEY" \
|
|
-F 'file=@/path/to/design.SchDoc' \
|
|
-F 'access=private' \
|
|
-F 'expires_in=3600'
|
|
```
|
|
|
|
Returns HTTP **202**:
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"id": "a15bd6b4e28487335c8f907d8f402b70fa92e1c31c943f20",
|
|
"status": "queued",
|
|
"viewer_url": "http://127.0.0.1:8000/view/a15bd6b4e28487335c8f907d8f402b70fa92e1c31c943f20#token=READ_TOKEN",
|
|
"viewer_token": "READ_TOKEN",
|
|
"status_url": "http://127.0.0.1:8000/api/v1/schematics/a15bd6b4e28487335c8f907d8f402b70fa92e1c31c943f20/status",
|
|
"access": "private",
|
|
"expires_at": 1793876400
|
|
}
|
|
```
|
|
|
|
Poll `status_url` with `X-Viewer-Token: READ_TOKEN` or `X-API-Key`. States are `queued`, `processing`, `completed`, `failed`. An accepted upload can later fail structure/version validation; 202 never asserts that a file rendered. The viewer URL may be opened immediately and polls until ready. Completed status includes `compatibility`, record/primitive/component counts, and diagnostics. `completed` means the supported rendering pipeline finished, not full format coverage.
|
|
|
|
| Endpoint | Purpose |
|
|
| --- | --- |
|
|
| `POST /api/v1/schematics/render` | Multipart upload with `file`, optional `access`, `expires_in` |
|
|
| `GET /api/v1/schematics/{id}/status` | State, expiry, diagnostics, failure details |
|
|
| `GET /api/v1/schematics/{id}/data` | Versioned normalized document, original properties, owners, primitives |
|
|
| `GET /api/v1/schematics/{id}/svg` | Safe vector rendering |
|
|
| `GET /view/{id}` | Full-viewport, schematic-only browser viewer |
|
|
| `GET /docs` | Self-contained API reference |
|
|
| `GET /openapi.json` | Machine-readable OpenAPI schema |
|
|
| `GET /healthz` | API/database liveness |
|
|
|
|
Error responses use `{"success": false, "error": {"code": "…", "message": "…"}}`. Missing/inaccessible private schematics return 404; expired authorized links return 410; unfinished artifacts 409; invalid files 422/415; excessive size 413; full queue or unavailable storage 503. See [API details](docs/API.md).
|
|
|
|
### Access and retention
|
|
|
|
`private` is the default. It creates a separate 256-bit read capability whose SHA-256 hash is stored in the database. The URL fragment is removed from the address bar after loading, retained in this tab's session storage, and sent only in `X-Viewer-Token`. Keep the original `viewer_url` if sharing/reopening in another browser. Status endpoints never return the token again. The API key grants service-wide access and must be kept server-side in external integrations.
|
|
|
|
`unlisted` is an explicit alternative: anyone with its 192-bit random identifier can read the result until expiry. There is no searchable gallery. Every access is checked on the server. Link lifetime is configurable and the worker removes expired database rows and artifacts in batches. After physical cleanup the endpoint returns 404. An already downloaded document cannot be revoked from a recipient's device. The reference viewer clears itself at expiry.
|
|
|
|
## Schematic Diff Viewer
|
|
|
|
Open `/compare` to upload an original and modified SchDoc, or call `POST /api/v1/comparisons` (alias `/api/compare`) with multipart fields `old_file` and `new_file` and the existing `X-API-Key`. It returns HTTP 202 with a private, expiring `/compare/{id}#token=…` URL. Poll `GET /api/v1/comparisons/{id}` using `X-Viewer-Token` until completed to receive summary counts and native-object differences.
|
|
|
|
The workspace displays original/modified SVGs side by side with synchronized zoom/pan, optional change boxes, filtering, changes-only mode, navigation and property details. It reuses the native parser, renderer, queue, resource-limited worker and local/S3 storage without new packages or database migrations. Net labels and wire geometry are compared; no compiled electrical equivalence is claimed. See [Schematic Diff documentation](docs/SCHEMATIC_DIFF.md) for matching rules, API examples, deployment limits and tests.
|
|
|
|
## Storage and configuration
|
|
|
|
Local files are private, written atomically with generated keys. Client filenames never determine storage paths. For private S3-compatible storage set the same `S3_BUCKET`, `S3_ENDPOINT_URL`, AWS credentials and region on API and workers; no bucket ACL or public URL is created. Prefer workload credentials/IAM and configure bucket encryption/lifecycle at the provider. On a multi-host deployment use S3; a Docker named volume is shared only on one host. Storage/database orphan reconciliation after a machine failure is an operational maintenance task; normal completion, retries and expiry cleanup are implemented.
|
|
|
|
| Environment | Default |
|
|
| --- | --- |
|
|
| `DATABASE_URL` | SQLite under `STORAGE_DIR`; Compose uses PostgreSQL |
|
|
| `STORAGE_DIR` | `./data` (`/data` in Compose) |
|
|
| `API_KEY` | Empty in local development; required in production |
|
|
| `PUBLIC_BASE_URL` | `http://127.0.0.1:8000` |
|
|
| `MAX_UPLOAD_BYTES` | 33,554,432 (32 MiB) |
|
|
| `PARSE_TIMEOUT_SECONDS` | 60 wall-clock seconds |
|
|
| `LINK_TTL_SECONDS` | 604,800 (7 days), maximum accepted `expires_in` |
|
|
| `MAX_PENDING_JOBS` | 100 active/unexpired jobs |
|
|
| `FONT_DIR` | Unset; optional administrator-managed webfont directory |
|
|
|
|
Malformed inputs are bounded by upload, OLE stream, record, point, ownership depth, raster expansion and subprocess resource limits. The parser never evaluates expressions, follows external image paths, runs Altium automation, or emits file-provided HTML/SVG/scripts. SVG markup is constructed using fixed tags and XML-escaped attributes/text; page CSP disables external resources and framing. No permissive CORS configuration is supplied.
|
|
|
|
The subprocess has a 768 MiB address-space limit, 60-second CPU ceiling and 128 MiB output-file ceiling, independently of the configurable wall-clock timeout. Resource limits contain malformed inputs; they are not a sandbox against an exploitable native image decoder. Use current patched dependencies and isolate the service from sensitive infrastructure. The example proxy does not add tenant accounts, per-client rate limiting or WAF rules; configure these to fit the deployment.
|
|
|
|
### Original fonts
|
|
|
|
Exact glyph shapes require the original fonts and appropriate licenses for web distribution. Set `FONT_DIR` to an administrator-controlled directory and create `fonts.json`:
|
|
|
|
```json
|
|
{
|
|
"Arial": [
|
|
{"file": "arial.woff2", "weight": "normal", "style": "normal"},
|
|
{"file": "arial-bold.woff2", "weight": "bold", "style": "normal"}
|
|
]
|
|
}
|
|
```
|
|
|
|
Mount that directory read-only on the API container. Only allowlisted local font files are served. Fonts are never extracted as executable resources from uploaded files. Browser fallback fonts are used when unavailable; the API always reports that font metrics need validation. SVG exports reference family names; they do not bundle proprietary fonts. Cairo validation requires the matching fonts installed on the validation host as well.
|
|
|
|
## Tests and fidelity validation
|
|
|
|
```bash
|
|
.venv/bin/python scripts/fetch-fixture.py
|
|
./scripts/check.sh
|
|
SCHDOC_SAMPLE='/path/to/a-real.SchDoc' ./scripts/check.sh
|
|
PYTHONPATH=backend .venv/bin/python scripts/corpus.py /path/to/real-schematics
|
|
.venv/bin/python scripts/validate.py /path/to/design.SchDoc \
|
|
--reference /path/to/trusted-altium-page.png
|
|
# With the API and worker running, test the real HTTP pipeline:
|
|
API_KEY="$API_KEY" .venv/bin/python scripts/smoke.py /path/to/design.SchDoc
|
|
```
|
|
|
|
The fixture downloader checksum-pins an actual upstream KiCad SchDoc. Unit-test circuits are explicitly synthetic and are never application demo data. Proprietary local designs are not redistributed or placed in the Docker image. The visual comparison tool emits SVG, PNG, heatmap, overlay and numerical differences, with nonzero exit on regression. It does not align/warp/crop a reference to conceal layout differences. Read [validation](docs/VALIDATION.md) for the release gate and current evidence.
|
|
|
|
## Architecture and PCB extension
|
|
|
|
See [architecture](docs/ARCHITECTURE.md). `app/documents.py` is the parser registry, the schema preserves source records separately from vector primitives, `app/render.py` is the SVG serializer, and workers/storage are independent of schematic record decoding. A future PCB adapter can introduce layer-aware primitives and an appropriate viewer through these boundaries; no PCB parser or PCB rendering is shipped here. `.PcbDoc` uploads are rejected.
|