archystream
ENRO

Extraction and automated description

Automated data extraction from documents

Archystream describes documents as they arrive and records where every value came from — so a reviewer checks a citation instead of re-reading the file.

How a document becomes a described record

  1. 1Arrival

    Received and secured

    Virus scan, SHA-256 digest, format identified from content rather than from the file name.

    seconds — the document is available and searchable
  2. 2Queue

    Queued for description

    Deeper analysis runs on a queue operators can inspect, retry and cancel per document.

    visible, retryable, never silently dropped
  3. 3Extract

    Read, not guessed

    Embedded properties, email headers, document text and OCR for images, then configurable rule packs for references, identifiers and dates.

    cached per stage — changing a rule re-runs that stage alone
  4. 4Evidence

    Every value cited

    Each value is recorded with its page, verbatim quotation, character range, method and confidence.

    a quotation that cannot be located is flagged, not accepted

Confirmed automatically

Facts read directly from the file: format, size, page count, digest.

Proposed for review

Anything inferred, queued by field so one decision covers many documents.

Always decided by a person

Retention, access restriction and other legal-consequence fields, at any confidence.

  1. 5Describe

    Projected into your fields

    Confirmed values populate the attributes your record series already uses, and the search index.

  2. 6Export

    Native, Dublin Core, PREMIS

    Fixity from the stored digest, stamped with the profile and vocabulary version used.

Times are from a 104-document reference corpus on a developer machine: available in seconds, fully described in minutes, and re-described in a fraction of that when only a rule changes.

Deterministic extraction on arrival

Format identified from content, embedded document properties harvested, email headers read for correspondents and threading, and rule packs matched for references, registration numbers, financial identifiers and dates, in Romanian, English and Dutch documents.

Every value cites the document

Each proposed value carries the page, the verbatim quotation and the character range it came from. A quotation that cannot be located in the stored text is flagged rather than accepted — the signal that something was inferred rather than read.

Values are proposals, not overwrites

Extracted values are recorded with their method, confidence and origin. Two sources may disagree without one silently replacing the other, and a reviewer's confirmation is recorded as a decision. Even “this document states no case reference” is a finding that can be recorded.

Confidence with limits that hold

Facts read directly from the file are confirmed automatically. Anything inferred is proposed for review, and fields with legal consequence — retention, access restriction — are never auto-confirmed at any confidence. A scan date is never quietly adopted as a record's creation date.

Review by field, not by document

Reviewers confirm one field across many documents in a single pass, with the citation beside each value and the source document alongside the form. Correcting a name once applies that correction wherever the same form appears.

Standards-based export

The same description exports to Dublin Core and PREMIS as well as the native schema, with the fixity digest taken from the stored checksum and each export stamped with the profile and vocabulary version that produced it.

Measured, not asserted

Accuracy reported per field and per document type

Automated description is only useful if its reliability is known. Archystream measures itself against frozen reference sets and reports the result field by field. A single overall accuracy figure hides the difference between a born-digital form and a poorly scanned one, and that difference determines how much review your team will actually do.

Reference sets are frozen, so one run can be compared with the run before it

Results are broken out by document type, never averaged into a single number

Fields with too few examples are reported as unmeasured rather than scored on noise

Correct abstention counts as a success, so silence is neither rewarded nor punished by accident

See Archystream with your own documents

Tell us about your record series and we will walk you through a portal configured for them, including the description quality we measure on documents like yours.

Or write to us at [email protected]