@@ -0,0 +1,43 @@
|
||||
# REST API contract
|
||||
|
||||
All routes are same-origin by default. Authentication for upload is `X-API-Key`. Private reads accept the same service key or `X-Viewer-Token`, returned only at creation. The service uses one integration key, not multi-tenant user authentication. Use a gateway for organization-specific account roles, key rotation, quotas and rate limits.
|
||||
|
||||
`POST /api/v1/schematics/render` accepts exactly one multipart file field `file`, with `.SchDoc` suffix (case insensitive). The client-provided content type is not trusted. The request body is capped before multipart spooling, and extracted file bytes are separately capped. Initial format identification accepts OLE signature or an ASCII SchDoc header; complete structural/version validation runs asynchronously in an isolated worker.
|
||||
|
||||
Optional fields:
|
||||
|
||||
- `access`: `private` (default) or `unlisted`.
|
||||
- `expires_in`: positive integer seconds, maximum `LINK_TTL_SECONDS`. Expiry starts at upload acceptance, including queue time.
|
||||
|
||||
HTTP 202 includes `id`, `status`, `viewer_url`, `status_url`, `expires_at` (Unix seconds), `access`, and a `viewer_token` for private uploads. `Location` points to status and `Retry-After: 1` provides a polling interval. Do not automatically retry a POST after a transport failure: each accepted request creates a new resource. Idempotency keys are not implemented.
|
||||
|
||||
`GET .../{id}/status` returns HTTP 200 for existing, accessible, unexpired jobs, including failed jobs with `success: false` and a stable `error.code`. Failed artifacts return HTTP 422. Status does not disclose the original filename, source bytes or read token. The returned base `viewer_url` omits the original token: retain the initial URL or append your retained read token as a fragment.
|
||||
|
||||
Completed responses add `schema_version`, `compatibility` (`partial` / `supported_subset`), `record_count`, `component_count`, `primitive_count`, and `diagnostics` (severity, code, message, record id/type, count). Current font metric warnings mean typical documents are explicitly partial. Warnings must be checked by applications where accurate reproduction matters.
|
||||
|
||||
`GET .../{id}/data` returns the document schema from `backend/app/models/schematic.py`:
|
||||
|
||||
- `schema_version`, `document_type`, `id`, `filename`, and `metadata` (source SHA-256, format/header, streams/counts).
|
||||
- `sheet` (original properties, page bounds/background), `bounds` (content bounds).
|
||||
- `components` / `symbols` / `objects` with ownership IDs, placement, raw properties, active visibility, pin anchors and primitive references.
|
||||
- `primitives` in source paint order (paths, text, sanitized embedded rasters).
|
||||
- `diagnostics` with compatibility failures/warnings. Raw properties are data, never executable DOM content.
|
||||
|
||||
Coordinates use 10 mil units, X to the right and Y downward (inverted once at parse time). The page begins at `(0, -height)` and ends at `(width, 0)`. Page dimensions are independent of content bounds. Consumers should not rotate/mirror placed component children again.
|
||||
|
||||
`GET .../{id}/svg` uses the same document primitives and page dimensions as the viewer; it has no interaction code. The interactive `/view/{id}` loads it and changes only its camera viewBox. Wheel zoom retains the point beneath the cursor; pointer events support drag/pinch. Double-click or Home resets fit; +/− and arrows provide keyboard control. Browser zoom, exact font availability and gesture behavior need actual browser validation.
|
||||
|
||||
Error code examples:
|
||||
|
||||
| HTTP | Code |
|
||||
| --- | --- |
|
||||
| 401 | unauthorized |
|
||||
| 404 | not_found (also private permission denied) |
|
||||
| 409 | not_ready |
|
||||
| 410 | link_expired (until retention cleanup) |
|
||||
| 413 | upload_too_large |
|
||||
| 415 | unsupported_file_type |
|
||||
| 422 | invalid_request / invalid_expiry / invalid_schematic / parse_timeout / parse_resource_limit |
|
||||
| 503 | queue_full / storage_unavailable / database_unavailable |
|
||||
|
||||
A worker crash is retried after its lease expires, at most twice total. A parse failure is terminal for that upload. No automatic retry of side-effecting API requests is performed. Data and SVG reads are immutable after a successful attempt; access/expiry remain server-authoritative on every read. HTTP caches are disabled to protect private drawings.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
C[External application] -->|authenticated multipart| A[FastAPI]
|
||||
A -->|generated private key| S[Local or S3 object storage]
|
||||
A -->|durable queued row| D[(PostgreSQL)]
|
||||
W[Independent worker replicas] -->|skip-locked lease| D
|
||||
W -->|bounded input| S
|
||||
W --> P[Resource-limited parser subprocess]
|
||||
P --> N[Native OLE / ASCII decoder]
|
||||
N --> G[Source records + versioned vector primitives]
|
||||
G --> V[SVG serializer]
|
||||
W -->|attempt-scoped JSON and SVG| S
|
||||
W -->|conditional completion| D
|
||||
B[Fullscreen browser] -->|read capability / expiry check| A
|
||||
A -->|JSON / vector artifact| B
|
||||
```
|
||||
|
||||
The API persists the source before admitting a queued database row. Only committed rows are claimable. An advisory lock serializes PostgreSQL admission limits. A failure before a database commit may leave a private orphan file; it cannot be accessed through any API without a row. Production operators should audit/reconcile orphans and configure S3 lifecycle policies.
|
||||
|
||||
The queue is in PostgreSQL, using `FOR UPDATE SKIP LOCKED` and a conditional update on attempt/status. SQLite uses the conditional update for development. A lease lasts the parser timeout plus 60 seconds for process startup/artifact storage. A crashed claim is retried once. Every attempt writes to a different key prefix and updates completion only if its attempt is still current. The physical source is immutable.
|
||||
|
||||
API and worker can scale independently. Worker parsing is serial per worker container; increase replicas for concurrency. Parser subprocesses inherit no API key, database URL or storage credentials, receive files in an ephemeral directory, and run under CPU/address-space/file-size/FD limits. The decoder never fetches document URLs or loads dynamic code. This resource boundary is defense in depth; hostile-file deployments may additionally isolate parsers with a hardened runtime/network namespace.
|
||||
|
||||
Local object paths are application-generated and validated. Files are mode 0600 in directories created as 0700. Compose uses the same persistent volume and UID for API/worker. For multiple hosts replace local storage with an S3-compatible private bucket. Database metadata and objects must be backed up consistently. Credentials are environment settings, not committed files or source-derived values.
|
||||
|
||||
The normalized document keeps original property maps, owner relationships and active parts separately from drawables. This is necessary for diagnostics and future parser improvements: the SVG is an output, not the only stored interpretation. Source paint order is preserved. Geometry is not laid out by a graph library.
|
||||
|
||||
`app/documents.py` registers format adapters. To support PCB later, add a PCB adapter and PCB-specific data/layer types, reuse queue/storage/capability APIs, and provide layer-aware renderer/viewer behavior. The current endpoint intentionally accepts only SchDoc. Existing optional layer/net schema fields reserve extension points; no saved PCB data is fabricated, and no PCB functionality is implemented.
|
||||
|
||||
The viewer is dependency-free JavaScript with a pure tested camera module. It manipulates one SVG viewBox, schedules updates with requestAnimationFrame, and delegates vector rendering to the browser. There is no raster redraw loop or symbol substitution. Large native records are processed off the API process. Browser SVG performance at very large primitive counts has not been benchmarked; viewport culling or a hybrid renderer may be warranted after profiling, while preserving the same primitive model.
|
||||
|
||||
Initial schema creation is serialized on startup. This is schema version 1; database evolution must use explicit reviewed migrations. Healthz checks API/database availability; it does not prove a worker is making progress. Monitor queued age, active leases, failures, storage capacity and replica health in production.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Compatibility and fidelity boundaries
|
||||
|
||||
This service is an independent, read-only native-format implementation. There is no official Altium SDK or Altium executable in its runtime. The current output is explicitly **partial**, with structured diagnostics. It must not be described as pixel-perfect, fully compatible with every Altium version, or certified for engineering signoff.
|
||||
|
||||
## File parsing
|
||||
|
||||
- OLE Compound File Binary containers with a `FileHeader` stream and a recognized format 5.0 header; also native ASCII records for that envelope.
|
||||
- `Storage` is decoded only for bounded, supported embedded raster assets. `Additional` records preserve stream-local indexes. Ambiguous cross-stream ownership is reported instead of inferred.
|
||||
- Little-endian length/kind record framing, original physical record indexes, source properties, UTF-8 companion properties, legacy Windows-1252 decoding.
|
||||
- Unknown binary payloads are preserved as base64, with diagnostics, rather than misinterpreted as text.
|
||||
- Major unknown headers, missing sheets, broken CFB/record lengths and files without supported geometry fail. Minor header revisions are reported, not treated as certified Altium application releases.
|
||||
- `.PcbDoc`, libraries without an instantiated schematic, encrypted/protected variants and unknown document envelopes are not supported.
|
||||
|
||||
## Geometry
|
||||
|
||||
Original component symbol primitives are extracted from the SchDoc itself. The renderer does not replace resistors, ICs or other components with generic icons. Ownership chains, active units/display modes, placements and fractional coordinates are retained. Placed children are not rotated a second time. Component rotation/mirroring remains in JSON; readable parameter glyphs do not inherit a placement mirror.
|
||||
|
||||
| Record IDs | Rendering behavior |
|
||||
| --- | --- |
|
||||
| 1, 2 | Component ownership, part selection; original pin segments, names/numbers; common bubble/clock decorations |
|
||||
| 4, 25, 32–34, 41 | Native labels, designators, parameter text and resolved stored parameter references |
|
||||
| 5–14 | Cubic Bezier sequences, polylines, polygons, ellipses, pie sectors, rounded rectangles, elliptical/circular arcs, lines, rectangles |
|
||||
| 15, 16 | Original sheet-symbol box and entry coordinates/names; entry arrow styles are incomplete and reported |
|
||||
| 17, 18 | Common power and off-sheet port styles using native style fields; unknown power markers omitted with diagnostics |
|
||||
| 22, 26, 27, 29, 37 | No-connect marks, bus/wire vertices, junction dots and bus entry lines |
|
||||
| 28, 209 | Text boxes, newlines, clipping, estimated wrapping (warning) |
|
||||
| 30 | Embedded BMP/PNG/JPEG/TIFF/GIF raster re-encoded as PNG after bounded decoding; multi-frame assets skipped |
|
||||
| 31 | Page background, 18 standard page presets, custom sizes and portrait orientation |
|
||||
| 39, 44–48 | Original template/model metadata retained; no library/resource execution |
|
||||
| 225 | Dashed annotation rectangle from observed records; not an official format guarantee |
|
||||
| Other | Preserved original properties and explicit unsupported-object diagnostics |
|
||||
|
||||
Source paint order is preserved. Explicit saved template lines/text render like other primitives. No schematic layout algorithm is used, and crossings do not fabricate junctions.
|
||||
|
||||
## Known visual gaps
|
||||
|
||||
1. **Automatic page decorations**: Altium-generated borders, zones and title blocks are not reconstructed. Explicit saved template primitives do render. Their omission is reported when enabled.
|
||||
2. **Typography**: Original font files and suitable web distribution licenses are required for exact glyph shapes. The optional administrator font allowlist supports regular/bold/italic variants. Browser fallback changes metrics. Font em conversion (currently 0.9 × stored size), default pin label offsets, baseline mapping and estimated wrapping require calibration against Altium reference exports. Font and pin metrics are always warned about.
|
||||
3. **Special text**: Per-character overbars, some custom text positioning, advanced compiled expressions, locale code pages other than Windows-1252/UTF-8 and unusual glyph encodings remain incomplete. Stored parameter references can resolve against extracted values; dates/times and missing data are not fabricated.
|
||||
4. **Decorations/styles**: IEEE symbols, uncommon pin decorations, line-end arrow decorations, some power marker styles, sheet-entry arrows and harness objects are not fully rendered. Unsupported power styles are omitted instead of replaced by another ground/power symbol. Common generated marker proportions and port text need reference validation. Non-solid line styles currently use a common dash sequence and are not certified to match every Altium dash pattern.
|
||||
5. **Images**: Only embedded, recognized raster files are decoded. Source filenames are treated solely as matching keys, never filesystem paths or network resources. Missing assets, multi-frame files, corrupt compression, pixel/decompression limits and ambiguous names produce warnings. No vector image/EMF/WMF conversion is provided.
|
||||
6. **Electrical scope**: Wire geometry, junctions and connection anchors are extracted; no Altium compiled netlist is created. Cross-sheet connectivity, repeated channels, bus expansion and harness semantics are not resolved. Hierarchical drawing objects on a single uploaded sheet are retained; child files are not fetched.
|
||||
7. **Layer order/performance**: Source primitive order is retained, but all proprietary editor-specific draw-order exceptions have not been characterized. Very large browser SVG scenes and mobile gestures require profiling on target devices.
|
||||
|
||||
An omitted object can be important. Applications must inspect `diagnostics`, not just `status: completed`. Original source bytes and raw properties are kept to support future decoder improvements.
|
||||
|
||||
## Proprietary dependencies and licensing
|
||||
|
||||
No proprietary component library is needed when a SchDoc contains instantiated symbol primitives. A document missing required saved geometry cannot be faithfully reconstructed without its original libraries or a licensed Altium workflow. This service does not invent that geometry. Exact fonts, external templates and absent graphics must be supplied by their owners under suitable licenses. Universal reference rendering may require licensed Altium Designer or an officially supported Altium rendering/export service; no such service is configured or impersonated here.
|
||||
|
||||
## Format research
|
||||
|
||||
Record/coordinate facts were checked against the [KiCad developer format reference](https://dev-docs.kicad.org/en/import-formats/altium/index.html) and [python-altium format notes](https://github.com/vadmium/python-altium/blob/master/format.md). The [Altium drawing-object documentation](https://www.altium.com/documentation/altium-designer/schematic/drawing-objects) and [connectivity documentation](https://www.altium.com/documentation/altium-designer/schematic/creating-circuit-connectivity) describe native graphic objects and power-port variations. These references do not constitute an official binary-format guarantee.
|
||||
|
||||
The initial independently written schematic decoding modules were reused from the user's local `/home/nearology/work/altium-render` checkout and extended here; that checkout was not modified. No GPL importer implementation has been copied into this project.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Rendering validation and release gate
|
||||
|
||||
Passing a parser test is not proof that a schematic matches Altium. The service separates format acceptance, supported-subset decoding, SVG generation, interactive behavior and trusted-reference fidelity.
|
||||
|
||||
## Tests
|
||||
|
||||
`./scripts/check.sh` runs bounds/framing/version/ownership/coordinate/part visibility tests, native-file integration when fixtures are present, safe SVG serialization, embedded raster expansion controls, upload/authentication/expiry/status/error flows, subprocess worker execution, queue capacity/claims/lease recovery, stale-worker fencing, expired artifact cleanup, and a botocore-validated S3 contract. Node's built-in runner checks sheet fit, camera invariants, point-under-cursor zoom and scale limits without frontend dependencies.
|
||||
|
||||
`python scripts/fetch-fixture.py` downloads an actual native OLE fixture from KiCad QA and enforces SHA-256 `e7dc42a4e983e6b716f6883eb0863fe47164d941636e11b1f2b5ec5d0143dca4`. It remains ignored and is not redistributed in Docker/source. `SCHDOC_SAMPLE` enables an additional actual design. Missing native files cause explicit skipped tests; they are not replaced by synthetic circuits. CI fetches the real fixture before tests.
|
||||
|
||||
`python scripts/corpus.py DIRECTORY` audits every SchDoc in a directory and writes source hashes, record counts, SVG sizes, failures and diagnostics. This detects format regressions across an actual design corpus without claiming visual equivalence.
|
||||
|
||||
## Trusted-reference procedure
|
||||
|
||||
1. Select representative designs containing rotated/mirrored and multi-part components; fractional placements; all pin orientations/decorations; buses/net labels; hierarchical ports; curves/fills; text/font variants; images; standard/custom/portrait sheets; dense and large pages. Include actual supported Altium versions and retain source SHA-256 values.
|
||||
2. In licensed Altium Designer, open each source and export a full-sheet reference PDF or PNG. Preserve original colors, dimensions and clipping. Record the exact Altium version, document settings, font versions, export options, page dimensions and export hash. Do not use a screenshot with toolbars or an arbitrary zoom/crop.
|
||||
3. Install licensed matching fonts on the validation host and configure equivalent webfonts for the browser. Rasterize PDF references with `pdftoppm` when necessary, retaining DPI/page size. No image-recognition step replaces parsing.
|
||||
4. Run `.venv/bin/python scripts/validate.py design.SchDoc --reference reference.png --output validation-output/design`. The render uses exactly the reference pixel dimensions and rejects incompatible aspect ratios. It does not fit to content or align away positional differences.
|
||||
5. Review `rendered.png`, `drawing.svg`, `difference.png`, `overlay.png` and `report.json`. The tool reports mean absolute RGB error (0–255) and the fraction of pixels whose maximum channel difference exceeds a tolerance. Default thresholds are MAE ≤ 2, changed fraction ≤ 1%, channel tolerance 12; tune and freeze them against accepted references, not failing candidates. It exits nonzero when thresholds fail. White/background-heavy images can hide local errors in global metrics, so manual/local-region review is mandatory.
|
||||
6. Compare native object anchors, pin endpoints, component symbol bounds, page dimensions, stroke widths, colors, rotations and textual baselines against measured reference values. Assert these coordinates independently in unit tests. Visually inspect actual browser renders, not only Cairo output, across target browser/OS/font combinations.
|
||||
7. Check initial sheet fit, drag, wheel, touch pinch, resize/aspect ratio, long-lived expiry, load/error states and a dense-file performance budget in connected desktop/mobile browsers. Loaded viewers must contain no toolbar, information panel, header or footer.
|
||||
8. Accept a format/version only after geometry checks, trusted visual comparisons and browser checks pass. Keep known failures as explicit compatibility restrictions. Pixel similarity alone does not prove electrical equivalence or universal pixel-perfect compatibility.
|
||||
|
||||
Even a passing comparison records `fidelity_verified: false`: automated image thresholds are a regression signal, not an automatic Altium-certification flag. Human review and the chosen supported-version corpus define a release's acceptance scope. Regenerate references only through a reviewed Altium export workflow.
|
||||
|
||||
## Evidence from this implementation session (2026-11-05)
|
||||
|
||||
- Final test run: 73 Python tests and 3 camera tests passed, including the native fixtures and a regression-harness test that detects altered pixels.
|
||||
- 13 actual local LTE 6 SchDoc files parsed and serialized to SVG without fatal errors; results and per-file diagnostics are in ignored `.runtime/corpus.json`. Counts range from 81 to 5,342 records. This is format coverage evidence, not visual acceptance.
|
||||
- The CPU sheet parsed 1,942 records and 25 components into 1,002 drawables; source SHA-256 `d6a73f11a4239eaf7e8f280ed35b82e9fd497a4680a60027cc93f266c5c006ae`.
|
||||
- The public Channel fixture parsed 99 records and 2 components into 34 drawables.
|
||||
- Both native sheets were rasterized with CairoSVG; the CPU PNG was visually inspected. Inspection found parameter glyph mirroring, which was corrected and regression-tested.
|
||||
- Eight simultaneous actual HTTP uploads to PostgreSQL produced eight unique IDs and all completed; this also passed with two worker replicas.
|
||||
- A built, non-root, read-only Compose API/worker with actual PostgreSQL completed the CPU upload/status/data/SVG flow over HTTP; private reads without a token returned 404. The published socket was confirmed at `127.0.0.1:8000`.
|
||||
- S3 request/response contracts are tested with botocore stubs; a live S3 provider has not been exercised.
|
||||
- No connected browser was available, so actual wheel/drag/touch behavior and browser font appearance remain unverified.
|
||||
- No trusted Altium export was supplied or available in this session. No reference comparison, visual equivalence, Altium release certification or pixel-perfect claim has been made.
|
||||
|
||||
The implementation is operationally testable; its known rendering gaps must be resolved and reference-tested before a production deployment requiring faithful Altium reproduction.
|
||||
مرجع در شماره جدید
مسدود کردن کاربر