# Human Value Observatory Research Library — Data Dictionary v0.4

## Recommended master format

Use JSON as the website-facing master file while the library is curated by hand. Keep a CSV export for spreadsheet review.

If the library grows to hundreds or thousands of entries, move the same schema into a database or content system such as SQLite, Airtable, Notion, Google Sheets, Supabase, or a headless CMS, then export JSON for the static website.

## Publication date fields

- `publication_date`: full publication date in ISO format, `YYYY-MM-DD`, where known.
- `publication_year`: year only, `YYYY`, where exact date is unknown or unnecessary.
- `date_precision`: `exact_date`, `month`, `year`, or `unknown`.
- `date_notes`: notes on uncertainty, edition, report year, revised date, or inferred date.
- `date_checked`: date when the publication date metadata was checked.

## Arena, lens and value fields

- `primary_arena`: the main Arena where the source is located. Arenas answer "where?"
- `secondary_arenas`: additional Arenas touched by the source.
- `primary_category`: legacy/general classification retained for compatibility.
- `human_value_tags`: value themes: the kind of value, activity, role or concern being investigated.
- `ai_impact_tags`: how AI may be changing the value, role, activity or concern.
- `lenses`: explicit canonical Human Value Lenses for Evidence Map matching.
- `relevant_lenses`: optional explicit Lens tags for source-level analysis. If absent, lens relevance may only be inferred provisionally from value themes.

Phase 1 keeps `primary_category` in place. Public display should prefer `primary_arena` where available and fall back to `primary_category` where arena metadata is missing.

Phase 3 Evidence Map matching uses explicit `lenses` / `relevant_lenses` values only. `human_value_tags` remain descriptive value themes and should not be used as a runtime substitute for explicit lens metadata.

Canonical Arena and Lens definitions live in `data/human-value-map-taxonomy.js` and are mirrored into the JSON `taxonomies` object. Each Arena and Lens has a stable `id`, display `name`, and `definition`. Arena x Lens descriptions are also mirrored into `taxonomies.arena_lens_descriptions` for later Evidence Map and Meaning Map phases.

Use this conceptual model throughout:

> Arenas locate the issue. Lenses interrogate the value. Sources provide evidence. Analysis creates meaning.

## Reserved source-quality fields

These fields are optional and reserved for later phases. They should not be used for Phase 1 reliability scoring, coverage calculations or confidence calculations.

- `scale`: future descriptor for the size or level of the affected context.
- `reach`: future descriptor for how widely a source's claims or evidence may apply.
- `evidenceQualityNote`: curator note about evidence limits, uncertainty or quality. This is not a reliability score.

## Access fields

- `url`: canonical public URL where possible.
- `publisher_owner`: organisation or rights holder most closely associated with the content.
- `access_granter`: platform or organisation that controls access in practice.
- `publisher_domain`: source domain.
- `access_status`: access classification.
- `access_model`: plain-language access model.
- `access_notes`: specific caveats such as paywall, account prompts, unstable video links, or institutional access.
- `last_checked`: date the URL/access fields were checked.

## Source text and retrieval fields

- `processing.textExtracted`: whether usable source text or curated source-grounding notes are available.
- `processing.chunked`: whether text has been split into retrievable chunks.
- `processing.searchIndexed`: whether chunks are included in keyword retrieval.
- `processing.embeddingIndexed`: reserved for future semantic retrieval.
- `processing.ragReady`: reserved for future retrieval-augmented generation.
- `text_quality`: `usable`, `limited`, `poor`, `metadata_only`, or `unknown`.
- `text_source_type`: `extracted_text`, `curated_source_notes`, `transcript`, `user_provided`, or `metadata_only`.
- `text_last_checked`: date text status was checked.
- `extraction_notes`: notes on extraction limits, chunking, or why text is unavailable.
- `copyright_reuse_status`: plain-language reuse status for stored text and excerpts.
- `citation_allowed`: whether the source can be cited/referenced in generated prompts.
- `quote_allowed`: whether short quotations/excerpts can be shown.

Static source chunks live under `data/source-text/`. The current Phase 4 seed chunks are curated source-grounding notes, not a full-text archive.

## Suggested access_status values

- `open_access`
- `paywalled_or_metered`
- `partial_or_uncertain_access`
- `needs_verification`
- `archived_or_dead_link`

## Website filter configuration

The JSON includes a `website_filter_config` object.

Use `default_visible_filters` for the simple public interface.

Use `advanced_revealable_filters` for a "Customise filters" panel where users can choose which metadata fields and tag groups to reveal.

Recommended behaviour:

1. Show default filters first.
2. Add a "Customise filters" button.
3. Let users reveal extra filters such as sector, publisher, evidence strength, framing, access model, and date precision.
4. Remember visible filter choices in localStorage.
5. Provide "Reset filters" and "Reset visible fields" buttons.

## Important note on reuse

Do not treat `open_access` as permission to reuse full text, images, charts, or video. It only means the item appears publicly accessible to read/watch. Reuse and republication rights should be tracked separately if the Observatory starts quoting, embedding, or republishing material.