AdvisoryAI Knowledge Search (AKS)

Why retrieval-first

AKS is a deterministic retrieval system for operational problem solving across Stella Ops docs, OpenAPI contracts, and Doctor checks. It is designed to work offline and does not require GPU-backed or hosted LLM inference for correctness.

LLMs can still be used as optional formatters later, but AKS correctness is grounded in source retrieval and explicit references.

Scope

Architecture

  1. Ingestion/indexing:
    • Markdown allow-list/manifest -> section chunks.
    • OpenAPI aggregate (openapi_current.json style artifact) -> per-operation chunks + normalized operation tables.
    • Doctor seed + controls metadata (including CLI-discovered Doctor check catalog projection) -> doctor projection chunks.
  2. Storage:
    • PostgreSQL tables in schema advisoryai (kb_doc at line 135, kb_chunk, …) via the single live migration src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations/001_v1_advisoryai_baseline.sql.
  3. Retrieval:
    • FTS (tsvector + websearch_to_tsquery) + optional vector stage.
    • Deterministic fusion and tie-breaking in KnowledgeSearchService.
  4. Delivery:
    • API endpoint: POST /v1/advisory-ai/search.
    • Index rebuild endpoint: POST /v1/advisory-ai/index/rebuild.

Unified-search architecture reference:

Data model

AKS schema tables:

Vector support:

Deterministic ingestion rules

Markdown

OpenAPI

Doctor

Ranking strategy

Implemented in src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchService.cs:

API contract

Rebuild

Localization runtime contract

Unified search interoperability

Unified search threat model (USRCH-POL-005)

Primary attack vectors and implemented mitigations:

Web behavior

Global search now consumes AKS and supports:

CLI behavior

AKS commands:

POST /v1/search/query additive response fields:

POST /v1/search/suggestions/evaluate:

Example:

curl -X POST http://127.0.0.1:10451/v1/search/suggestions/evaluate \
  -H "Content-Type: application/json" \
  -H "X-StellaOps-TenantId: test-tenant" \
  -H "Authorization: Bearer <token-with-advisory-ai:operate>" \
  -d '{
    "queries": ["database connectivity", "oidc readiness"],
    "ambient": {
      "currentRoute": "/ops/operations/doctor"
    }
  }'

Notes:

Output:

Test/benchmark strategy

Implemented benchmark framework:

Tests:

Unified-search quality benchmarks:

Dedicated AKS test DB

Compose profile:

Init script: none — the extensions are created by the service’s own startup migration. (The previously cited devops/compose/postgres-init/advisoryai-knowledge-test/01_extensions.sql does not exist; verified 2026-07-12. postgres-init/ holds only 00-v1-baseline.sql + _archived/.)

Example workflow:

docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d
stella advisoryai sources prepare --json
stella advisoryai index rebuild --json
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj

Search improvement sprints (G1–G10) — testing infrastructure guide

Ten search improvement sprints (SPRINT_20260224_101 through SPRINT_20260224_110) were implemented as a batch. This section documents how to set up infrastructure and run the full test suite.

Sprint inventory

SprintGapTopicModule(s)
101G5FTS English stemming + trigram fuzzyAdvisoryAI (backend)
102G1ONNX semantic vector encoderAdvisoryAI (backend)
103G2Cross-domain live-data adaptersAdvisoryAI (backend)
104G3LLM-grounded synthesis engineAdvisoryAI (backend)
105G4Search onboarding + guided discovery + “Did you mean?”FE + AdvisoryAI
106G6Search personalization (popularity boost, role-based bias, history)AdvisoryAI + FE
107G7Search → Chat bridge (“Ask AI” button)FE
108G8Inline result previews (expandable entity cards)AdvisoryAI + FE
109G9Multilingual search (de/fr/es/ru FTS, language detection, localized doctor seeds)AdvisoryAI + FE
110G10Search feedback loop (thumbs up/down, quality dashboard, query refinements)AdvisoryAI + FE

Test projects and files

All backend tests live in a single test project:

src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj

Key test files added by the search sprints:

FileCoverageType
Integration/UnifiedSearchSprintIntegrationTests.csAll 10 sprints (87 tests) — endpoint auth, domain filtering, synthesis, suggestions, role-based bias, multilingual detection, feedback validationIntegration (WebApplicationFactory)
Integration/KnowledgeSearchEndpointsIntegrationTests.csAKS endpoints: auth, search, localization, rebuildIntegration (WebApplicationFactory)
KnowledgeSearch/FtsRecallBenchmarkTests.csG5-005: FTS recall benchmark (12 tests, 34-query fixture)Benchmark
KnowledgeSearch/FtsRecallBenchmarkStore.csIn-memory FTS store simulating Simple vs English modesTest harness
KnowledgeSearch/SemanticRecallBenchmarkTests.csG1-004: Semantic recall benchmark (13 tests, 48-query fixture)Benchmark
KnowledgeSearch/SemanticRecallBenchmarkStore.csIn-memory vector store with cosine similarity searchTest harness
UnifiedSearch/UnifiedSearchServiceTests.csG8: Preview generation (7 tests)Unit

Test data fixtures (auto-copied to output via TestData/*.json glob in .csproj):

Prerequisites to run

Detailed infrastructure setup guide: src/AdvisoryAI/__Tests/INFRASTRUCTURE.md — covers 4 tiers (in-process, live database, ONNX model, frontend E2E) with exact Docker commands, connection strings, extension requirements, and config examples.

No external infrastructure needed for the in-process test suite. All integration tests use WebApplicationFactory<Program> with stubbed services. Benchmarks use in-memory stores. No PostgreSQL, no Docker, no network access required.

Run the full suite:

dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v normal

Targeted runs in this project use xUnit v3 / Microsoft.Testing.Platform. Do not use VSTest dotnet test --filter ... syntax here; use xUnit pass-through or the built test executable instead.

Run only the search sprint integration tests:

dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
  -- --filter-class StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests

Run only the FTS recall benchmark:

dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
  -- --filter-class StellaOps.AdvisoryAI.Tests.KnowledgeSearch.FtsRecallBenchmarkTests

Run only the semantic recall benchmark:

dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
  -- --filter-class StellaOps.AdvisoryAI.Tests.KnowledgeSearch.SemanticRecallBenchmarkTests

For live database tests (e.g., full AKS rebuild + query against real Postgres with pg_trgm/pgvector):

# Start the dedicated AKS test database
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d

# Wait for health check
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml ps

# Start the local AdvisoryAI service against that database
export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile

# In a second shell, rebuild the live corpus in the required order
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \
  -H "X-StellaOps-Scopes: advisory-ai:admin" \
  -H "X-StellaOps-TenantId: test-tenant" \
  -H "X-StellaOps-Actor: local-search-test"
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
  -H "X-StellaOps-Scopes: advisory-ai:admin" \
  -H "X-StellaOps-TenantId: test-tenant" \
  -H "X-StellaOps-Actor: local-search-test"

# Run tests with the Live category (requires database)
dotnet build "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v minimal
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/bin/Debug/net10.0/StellaOps.AdvisoryAI.Tests.exe \
  -trait "Category=Live" -reporter verbose -noColor

Notes:

CLI setup in a source checkout

Do not assume stella is already installed on the machine running local AdvisoryAI work. From a repository checkout, either run the CLI through dotnet run or publish a local binary first.

Quick references:

Local examples:

# Run directly from source without installing to PATH
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json

# Publish a reusable local binary
dotnet publish "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -c Release -o ".artifacts/stella-cli"

# Windows
.artifacts/stella-cli/StellaOps.Cli.exe advisoryai index rebuild --json

# Linux/macOS
./.artifacts/stella-cli/StellaOps.Cli advisoryai index rebuild --json

If the CLI is not built yet, the equivalent HTTP endpoints are:

Standard browser acceptance lane for live search:

cd src/Web/StellaOps.Web
npm run test:e2e:search:live

This command is now the standard local search-browser setup because it:

Current live verification coverage:

Or use the full CI testing stack:

docker compose -f devops/compose/docker-compose.testing.yml --profile ci up -d

Database extensions required for live tests

The AdvisoryAI startup baseline src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations/001_v1_advisoryai_baseline.sql enables them itself, so a fresh DB converges with no manual init step (per AGENTS.md §2.7):

The Postgres image must simply permit those extensions. If setting up a custom test database:

CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;

Migrations required for search sprints

The search sprints originally shipped several migrations under src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations/. They were collapsed into the single live baseline 001_v1_advisoryai_baseline.sql; the filenames below are provenance of the folded-in DDL, not live files (they sit under Storage/Migrations/_archived/pre_1.0/mig061/, excluded from embedding). Note the archived English-FTS file is actually named 004_fts_english_trgm.sql.

Folded-in pre-1.0 source (archived)SprintContent
004_fts_english_trgm.sqlG5 (101)body_tsv_en tsvector column + GIN index, pg_trgm extension + trigram indexes
005_search_feedback.sqlG10 (110)search_feedback (baseline line 445) + search_quality_alerts (line 495) tables
005_search_analytics.sqlG6 (106)search_events (baseline line 416) + search_history tables
007_multilingual_fts.sqlG9 (109)body_tsv_de, body_tsv_fr, body_tsv_es, body_tsv_ru tsvector columns + GIN indexes
008_search_self_serve_analytics.sqlAI-SELF-004session_id, answer_status, answer_code analytics columns plus self-serve indexes

The baseline is idempotent (IF NOT EXISTS guards) and is applied automatically by AddStartupMigrations at service startup.

Frontend tests

Frontend changes span src/Web/StellaOps.Web/. To run Angular unit tests:

cd src/Web/StellaOps.Web
npm install
npm run test:ci

For E2E tests (requires the full stack running):

cd src/Web/StellaOps.Web
npx playwright install
npm run test:e2e

Relevant E2E config: src/Web/StellaOps.Web/playwright.e2e.config.ts.

InternalsVisibleTo

The production assembly StellaOps.AdvisoryAI grants InternalsVisibleTo to StellaOps.AdvisoryAI.Tests (see src/AdvisoryAI/StellaOps.AdvisoryAI/Properties/AssemblyInfo.cs). This allows tests to access internal types including:

Key interfaces to stub in integration tests

InterfacePurposeTypical stub behavior
IKnowledgeSearchServiceAKS searchReturn hardcoded results per query
IKnowledgeIndexerAKS index rebuildReturn fixed summary counts
IUnifiedSearchServiceUnified searchReturn entity cards with domain filtering
IUnifiedSearchIndexerUnified index rebuildReturn fixed summary
ISynthesisEngineAI synthesisReturn template-based synthesis
IVectorEncoderEmbedding generationUse DeterministicHashVectorEncoder or EmptyVectorEncoder
IKnowledgeSearchStoreFTS/vector storageUse DeterministicBenchmarkStore or FtsRecallBenchmarkStore

Test categories and filtering

Use [Trait("Category", TestCategories.XXX)] to categorize tests. Key categories:

Filter examples:

# All except Live
dotnet test ... --filter "Category!=Live"

# Only integration
dotnet test ... --filter "Category=Integration"

# Specific test class
dotnet test ... --filter "FullyQualifiedName~FtsRecallBenchmarkTests"

Localized doctor seeds

Doctor check content is available in 3 locales:

The KnowledgeIndexer.IngestDoctorAsync() method auto-discovers locale files via DoctorSearchSeedLoader.LoadLocalized() and ingests locale-tagged chunks alongside English. This enables German/French FTS queries to match doctor check content.

Configuration options added by search sprints

All in KnowledgeSearchOptions (src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchOptions.cs):

OptionDefaultSprintPurpose
FtsLanguageConfig"english"G5Primary FTS text search config
FuzzyFallbackEnabledtrueG5Enable pg_trgm fuzzy fallback
MinFtsResultsForFuzzyFallback3G5Threshold for fuzzy activation
FuzzySimilarityThreshold0.3G5pg_trgm similarity cutoff
VectorEncoderType"hash"G1"hash" or "onnx"
OnnxModelPath"models/all-MiniLM-L6-v2.onnx"G1Path to ONNX model file
LlmSynthesisEnabledfalseG3Enable LLM-grounded synthesis
SynthesisTimeoutMs5000G3LLM synthesis timeout
LlmAdapterBaseUrlnullG3LLM adapter service URL
LlmProviderIdnullG3LLM provider selection
PopularityBoostEnabledfalseG6Enable click-weighted ranking
PopularityBoostWeight0.05G6Popularity boost factor
RoleBasedBiasEnabledtrueG6Enable scope-based domain weighting
SearchQualityMonitorEnabledtrueG10Enable periodic quality-alert refresh
SearchQualityMonitorIntervalSeconds300G10Quality-alert refresh cadence
SearchAnalyticsRetentionEnabledtrueG10Enable automatic analytics/feedback/history pruning
SearchAnalyticsRetentionDays90G10Retention window for search analytics artifacts
SearchAnalyticsRetentionIntervalSeconds3600G10Retention pruning cadence
FtsLanguageConfigs{}G9Per-locale FTS config map

Unified-search options (UnifiedSearchOptions, src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchOptions.cs):

Known limitations and follow-ups