ironhub-enrichment-server documentation

IronHub Enrichment Server

Backend API for the IronHub AI Product Enrichment platform — a NestJS application that ingests product data via file upload (Excel/CSV) or API, sends it to an external AI enrichment service, and exposes the enriched results through a RESTful API with pagination, filtering, approval workflows, and Swagger documentation.

Features

  • NestJS 11 — modular architecture with dependency injection
  • TypeScript strict mode — type-safe across the entire codebase
  • PostgreSQL + TypeORM — relational database with migration-based schema management
  • Bulk import pipeline — upload Excel/CSV files, map columns, and queue items for enrichment
  • AI enrichment integration — sends products to an external Python enrichment service
  • Approval workflow — field-level accept/reject/edit with bulk approval support
  • Re-enrichment & versioning — re-run enrichment with overrides, automatic version tracking
  • Generic query system — pagination, sorting, full-text search, and advanced filtering on all list endpoints
  • Swagger / OpenAPI docs — auto-generated at /docs (toggle via ENABLE_DOCS)
  • Compodoc — application architecture documentation served at /docs/compodoc
  • Global exception handling — consistent JSON error responses with structured logging
  • Request logging interceptor — logs method, URL, and response time for every request
  • CORS — configurable allowed origins with runtime overrides for local development
  • Input validation — all request bodies validated with class-validator and class-transformer

Project Structure

Example :
src/
├── app.module.ts              # Root module — imports all feature modules
├── app.controller.ts          # Health check, status page, favicon endpoints
├── app.service.ts             # Server metadata & uptime
├── main.ts                    # Bootstrap — pipes, filters, CORS, Swagger setup
├── data-source.ts             # TypeORM DataSource for migrations CLI
├── common/
│   ├── dto/                   # Generic query, pagination, filter DTOs
│   ├── filters/               # AllExceptionsFilter
│   ├── interceptors/          # LoggingInterceptor
│   └── helpers/               # QueryBuilderHelper (filtering, sorting, search)
├── config/
│   ├── configuration.ts       # Typed environment configuration
│   └── validation.schema.ts   # Joi schema for env validation
├── database/
│   ├── database.module.ts     # TypeORM module registration
│   └── entities/              # TypeORM entities (ImportBatch, ImportItem, EnrichmentJob, EnrichmentAttempt, EnrichmentResult)
├── migrations/                # TypeORM migrations (schema changes)
└── modules/
    ├── auth/                  # Auth module (placeholder)
    ├── enrichment/            # Core enrichment module
    │   ├── enrichment.controller.ts
    │   ├── enrichment.service.ts
    │   ├── enrichment.module.ts
    │   └── dto/               # Enrichment & approval DTOs
    └── integrations/          # External integrations (IronHub)

Getting Started

Prerequisites

  • Node.js v18+
  • PostgreSQL 14+ (local or remote)
  • npm (ships with Node.js)

Installation

Example :
npm install

Environment Configuration

  1. Copy the example environment file:
Example :
cp .env.example .env
  1. Update .env with your values:
Example :
NODE_ENV=development
PORT=3001
CORS_ORIGIN=http://localhost:3000

# API
API_PREFIX=api
API_VERSION=v1

# Database (PostgreSQL)
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_USERNAME=postgres
DATABASE_PASSWORD=postgres
DATABASE_NAME=ironhub_enrichment
DATABASE_SSL=false
DATABASE_SSL_REJECT_UNAUTHORIZED=false

# External AI enrichment service
PYTHON_URL=http://localhost:5000

# Documentation
ENABLE_DOCS=true

Tip — CORS for local dev against staging: Append your dev origin to CORS_ORIGIN (comma-separated), or set EXTRA_CORS_ORIGINS=http://localhost:3000, or set ALLOW_CORS_LOCALHOST=true. Restart the service after changing.

Database Setup

Example :
# Run migrations to create / update schema
npm run migration:run

# (Optional) Reset database — local dev only, drops and recreates public schema
npm run db:reset

Running the Application

Example :
# Development (watch mode)
npm run start:dev

# Production build
npm run build:prod     # generates Compodoc docs, builds NestJS, copies static assets
npm run start:prod

The API will be available at http://localhost:3001/api.

API Endpoints

Core endpoints

Method Path Description
GET / HTML status page (name, version, environment, uptime)
GET /api JSON health-check (for monitoring tools)

Enrichment endpoints (prefixed with /api)

Method Path Description
POST /enrichment/bulk Upload Excel/CSV file or JSON products for bulk enrichment
GET /enrichment/batch/:batchId/status Get batch processing status & progress
GET /enrichment/batch/:batchId/items List items in a batch with status & results
GET /enrichment/jobs Paginated list of enrichment jobs
GET /enrichment/results Paginated, filterable enrichment results
GET /enrichment/results/stats Approval statistics (waiting / accepted / rejected)
GET /enrichment/results/:id Single enrichment result detail
PATCH /enrichment/results/:id/approve Approve or reject a result (field-level)
POST /enrichment/results/:id/re-enrich Re-enrich a result with optional overrides
POST /enrichment/results/bulk-approve Bulk-approve results by confidence threshold

Documentation endpoints

Path Description
/docs Swagger / OpenAPI interactive API docs
/docs/compodoc Compodoc application architecture docs

Testing

Example :
# Unit tests
npm run test

# Unit tests in watch mode
npm run test:watch

# E2E tests
npm run test:e2e

# Test coverage
npm run test:cov

Code Quality

Example :
# Lint (with auto-fix)
npm run lint

# Format (Prettier)
npm run format

# Format check (CI)
npm run format:check

Database Migrations

Example :
# Generate a new migration from entity changes
npm run migration:generate -- src/migrations/<MigrationName>

# Run pending migrations
npm run migration:run

# Revert last migration
npm run migration:revert

Database Reset (dev-only)

Warning: This destroys all data. For local development only.

Example :
# Interactive (recommended)
npm run db:reset

# Skip backup
npm run db:reset -- --no-backup

# Non-interactive
npm run db:reset -- --yes

# Force even in production (NOT RECOMMENDED)
npm run db:reset -- --force --yes

Build & Documentation

Example :
# Full production build (Compodoc + NestJS + copy static assets)
npm run build:prod

# Generate Compodoc docs only
npm run docs:compodoc

# Serve Compodoc docs locally (http://127.0.0.1:8080)
npm run docs:compodoc:serve

Architecture Highlights

  • Entities — 5 TypeORM entities: ImportBatch, ImportItem, EnrichmentJob, EnrichmentAttempt, EnrichmentResult
  • Approval flow — results start as waiting_for_approval → reviewers accept/reject at the field level → accepted results can be published
  • Versioning — re-enrichment increments the version column and links to the previous result via previousVersionId
  • Generic query — BaseQueryDto provides pagination, sort, search, and a JSON filters parameter supporting typed operators (string, number, date)
  • QueryBuilderHelper — reusable TypeORM utility for applying filters, sorting, search, and pagination to any entity query

License

UNLICENSED (private)

results matching ""

    No results matching ""