dc-api-v2

Local development setup

env.json

The env.json file contains environment variable values for the lambda functions defined in the API for use in local development. You can create an env.json file containing the values to run the API against your dev data by running:

make env.json

If the file already exists, it will not be overwritten unless you include -B in the make command.

Running the API locally

To start the API in development mode, first make sure you have the correct version of the AWS SAM command line utility installed:

asdf install aws-sam-cli

Then run the following command:

make serve

The API will be available at:

• https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002
  • Don't forget to open port 3002 if you want to access it remotely

⚠️ Note the above URLs (which point to your local OpenSearch instance) need full endpoints to resolve. For example:

• https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/search
• https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/collections

View supported endpoints Questions? View the production API documentation

Chaos middleware

The API supports simulated network effects (errors and delays) for local testing via the CHAOS_CONFIG environment variable. If the variable is absent the middleware is disabled entirely.

Set it to an inline JSON array:

export CHAOS_CONFIG='[
  { "pattern": "/works/:id", "effect": "error", "status": 500, "chance": 0.3 },
  { "pattern": "/auth/whoami", "effect": "delay", "ms": 500 },
  { "pattern": "/file-sets/*", "effect": "delay", "ms": [100, 800] }
]'

Or set it to the path of a JSON file containing the same array:

export CHAOS_CONFIG=/path/to/chaos.json

Each rule has a pattern (matched against the request path) and an effect:

All matching rules are evaluated in order. Delay rules accumulate; an error rule short-circuits the request only when it fires — otherwise evaluation continues to the next rule.

Example workflows

Meadow

View and edit information about a specific Work in the Index.

1. Open a local Meadow instance.
2. Find an id of a Work you'd like to inspect in the API.
3. View JSON response at https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]
4. View IIIF Manifest JSON response at https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]?as=iiif

IIIF content search

IIIF Presentation responses expose IIIF Content Search 2.0 services for transcription annotations:

• Work manifests include a SearchService2 entry for https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]/search?as=iiif
• File set canvases include a SearchService2 entry for https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/file-sets/[FILE_SET_ID]/search?as=iiif

To search transcription text, include a non-empty q parameter:

curl "https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]/search?as=iiif&q=[QUERY]"
curl "https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/file-sets/[FILE_SET_ID]/search?as=iiif&q=[QUERY]"

Both endpoints return a IIIF AnnotationPage whose items target the matching work canvas or file set canvas. Requests without as=iiif or a non-empty q return 400.

For help debugging/inspecting, JavaScript console messages are written to: dc-api-v2/dc-api.log

DC

Develop against changes to the API.

1. Before starting the DC app, temporarily change the port number in dc-nextjs/server.js from default 3000 to something like 3003.
2. Open the port so it can be accessed in the browser.

sgport open all 3003

3. Point to the proxy URL and start DC app (in your /environment/dc-nextjs shell)

export NEXT_PUBLIC_DCAPI_ENDPOINT=https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002
bun run dev

Access the app in a browser at: https://USER_PREFIX.dev.rdc.library.northwestern.edu:3003/

Running the API locally with state machine + lambdas (needed for AV download route)

# From the repo root
cd dc-api-v2

# Start the API + step function and associated lambdas
make start-with-step

# Open a second terminal and create the state machine
make state-machine

Deploying a development branch

There are two ways to deploy a development branch: make deploy and make sync. The differences are:

• Changes: deploy deploys a static stack, and requires another deploy to update it. sync watches for changes in realtime.
• Dependencies: deploy uses the apiDependencies resource defined in the template for dependencies, while sync uses the AWS SAM CLI's built-in development dependency logic.

Either way, the resulting stack will be accessible at https://dcapi-USER_PREFIX.rdc-staging.library.northwestern.edu.

An existing sync stack can be reused by running make sync again, or by running make sync-code to only sync code changes (no infrastructure/template changes).

samconfig.*.yaml

Both methods involve a samconfig.USER_PREFIX.yaml file. This file, with default values, can be created by running (for example):

make samconfig.mbk.yaml

This will create a configuration to stand up the default stacks in both deploy mode (API, AV Download, and Chat) and sync mode (Chat only). To deploy a different combination of features, specify them using the WITH option:

make samconfig.mbk.yaml WITH=API,DOCS

Available features are: API, AV_DOWNLOAD, CHAT, and DOCS.

⚠️ Be very careful including the API in sync mode as every change within /api will take a long time to deploy.

As with the env.json file, make will not overwrite an existing file unless you include -B.

Tearing down a development stack

sam delete --stack-name dc-api-USER_PREFIX

Writing Documentation

API documentation is automatically regenerated and deployed on pushes to the staging and production branches. The documentation is in two parts:

Regular Docs

The docs directory contains a standard mkdocs project, which can be edited using the same tools and format as the main Repository Documentation.

In a nutshell:

1. Clone this project into a working directory (which you probably already have).
2. Edit the Markdown files in the docs/docs directory.
3. To run mkdocs locally and preview your work:
   shell

   sgport open all 8000
   make serve-docs
   

   Docs will be accessible at http://USER_PREFIX.dev.rdc.library.northwestern.edu:8000/

OpenAPI/Swagger Docs

We also maintain an OpenAPI Specification under the docs directory in spec/openapi.yaml. When mkdocs is running, the Swagger UI can be found at http://USER_PREFIX.dev.rdc.library.northwestern.edu:8000/spec/openapi.html. Like the rest of the documentation, changes to the YAML will be immediately visible in the browser.

The existing spec files (openapi.yaml, types.yaml, and data-types.yaml) are the best reference for understanding and updating the spec. It's especially important to understand how openapi.yaml uses the $ref keyword to refer to reusable elements defined in types.yaml, and how types.yaml pulls model schemas from data-types.yaml.

For an in-depth look, or to learn how to define things for which there aren't good examples in our spec, refer to the full OpenAPI documentation.

Build Artifacts

openapi.html renders the Swagger UI directly from the unmodified openapi.yaml. In addition, the build process generates a JSON copy of the spec using the OpenAPI Generator CLI. In order to make sure the spec is valid before checking it in, run:

bun run validate-spec

This check is also part of the CI test workflow, so an invalid spec file will cause the branch to fail CI.

DC API Typescript NPM package

Typescript types for the schemas (Works, Collections, FileSets) are automatically published to the nulib/dcapi-types repo on deploys.

• If a deploy to the deploy/staging branch contains changes to the docs/docs/spec/data-types.yaml file, new types are generated and a commit is made to the staging branch of nulib/dcapi. This is intended to be for local testing by NUL devs against the private staging API.
• If a deploy to production (main branch) contains changes to the docs/docs/spec/data-types.yaml file, new types are generated and a PR is opened into the main branch of nulib/dcapi-types. Also, an issue is created in nulib/repodev_planning_and_docs to review the PR and publish the types package (manually).

Versioning

The current API version is maintained in several different project files. To increment the version, use

make version BUMP=<major|minor|patch>

If you don't specify a BUMP value, the command will simply print the current version.