Chapter 7: Testing and Validation — Proving Your Domain Works

"Anyone can build a domain that works with perfect data in ideal conditions. The goal is a domain that works correctly with every data scenario, every edge case, and every combination of fields that a real organization will throw at it."

Why Testing Is Not Optional

You have built something. The CSV files exist, the domain-config.json is written, the generator script ran successfully, the README is polished and ready. It is tempting at this point to package everything up and publish.

Do not do that yet.

The gap between "the generator ran without errors" and "the domain works correctly for real users" is wider than it appears. A generator error means the JavaScript broke. A domain validation failure means the platform cannot load your config. But there is an entire third category — silent failures — where everything appears to work, a document generates successfully, and the output is wrong in a way that will not be caught until a real attorney sends a client an invoice with a blank billing section, or a co-op administrator mails a family an emergency contact card with another family's information on it.

Silent failures are the most dangerous kind because users do not always know they are seeing incorrect output. They trust the document because the system generated it. Your testing process exists to catch every silent failure before any user ever sees it.

Testing a domain has five phases: domain loading validation, template loading verification, data relationship testing, document generation testing, and edge case testing. Each phase catches a different class of problem. All five are required.

Phase 1: Domain Loading Validation

Before testing templates or generating documents, confirm that the platform can load your domain package without errors. This validates your domain-config.json, file structure, and data files.

Loading Your Domain

Open Microsoft Word with the DataPublisher add-in active.

1. Go to the Data tab
2. Choose Data Sets
3. Go to Domain Template Catalog
4. Find your domain in the catalog (if testing a pre-release version, it may appear in a "Draft" or "Testing" section)
5. Click Clone Template

The platform will: - Validate your domain-config.json structure - Check that all referenced tables exist in csv-data/ - Verify all template files exist in word-templates/ - Load your sample CSV data - Import the domain into your workspace

What Success Looks Like

Successful load: You'll see a success message: "✓ Domain cloned successfully: Legal Services (20 templates, 10 tables)". Your domain appears in the Data Sets list with all tables showing record counts.

Configuration error: If domain-config.json has a structural issue, you'll see an error message like:

✗ Domain validation failed: Table 'billing_entries' references 
unknown table 'invoices' in foreign key 'InvoiceID'

Read the error message precisely. It names the exact table and field where the problem was found. Fix it in domain-config.json and try cloning again.

Missing file error: If a template file referenced in domain-config.json doesn't exist, you'll see:

✗ Template file not found: word-templates/Template-03-Invoice.docx

Verify the filename in domain-config.json matches the actual file in word-templates/.

CSV data error: If a CSV file has a format error, you'll see which file and which row caused the problem. Common issues: missing column headers, inconsistent column counts between rows, or special characters not properly escaped.

Verifying Data Loaded Correctly

Once your domain clones successfully, verify your sample data:

1. Stay in the Data tab
2. Click on each table in your Data Sets list
3. Verify the record count matches what you created:
4. firms: 3 records
5. practice_areas: 9 records
6. clients: 10 records
7. attorneys: 9 records
8. matters: 10 records
9. hearings: 6 records
10. deadlines: 10 records
11. billing_entries: 14 records
12. invoices: 4 records
13. documents_filed: 6 records

If any table shows 0 records when it should have data, click View Data to inspect it. The issue is usually a CSV formatting problem: missing headers, extra commas, or inconsistent quoting.

Phase 1 Pass Criteria

• Domain clones successfully from the catalog without errors
• All tables appear in Data Sets with correct record counts
• No error messages about missing files or invalid configuration

Do not proceed to Phase 2 until the domain loads cleanly.

Phase 2: Template Loading Verification

Phase 2 verifies that every template appears correctly and presents its data accurately before any document is generated.

The Visual Inspection Checklist

With your domain loaded in DataPublisher, navigate to the Templates tab and work through this checklist for every one of the 20 templates:

Checklist item 1: Template appears in the correct category Every template should appear under the category defined in its documentTypes entry. If a template appears in the wrong category, the category field in domain-config.json is incorrect.

Checklist item 2: Template description is readable and accurate Click each template and read the description that appears in the information panel. Confirm it matches what you wrote in domain-config.json and that it describes the template accurately. A description mismatch here usually means you copied a template definition and forgot to update the description.

Checklist item 3: Primary table selector shows correct records For each template, the primary table selector should populate with records from the correct table. The Engagement Letter's primary table is matters — the selector should show matter records. The Invoice's primary table is invoices — the selector should show invoice records.

If the selector shows records from the wrong table, the primaryTable field in the template's domain-config.json entry is incorrect.

Checklist item 4: Record count matches expectations The selector should show the number of records you expect from each primary table. If it shows fewer records than you created, there may be a filtering issue or a loading error for specific records. If it shows more records than you created, there may be a duplicate import.

Checklist item 5: Related table data previews correctly For templates with multiple related tables, the data preview panel should show fields from all related tables correctly. Hover over or select a record and confirm that related data — client name for a matters-based template, attorney name, firm details — appears correctly in the preview.

This checklist takes approximately 20 to 30 minutes for a 20-template domain. Every minute is worth it.

Phase 2 Pass Criteria

• All 20 templates appear in the correct categories
• All template descriptions are accurate
• Every primary table selector shows the correct records
• Related table data previews correctly for every template

Phase 3: Data Relationship Testing

Phase 3 is where you verify that the foreign key relationships you defined in Chapter 4 actually work when data flows through them. This is the phase that catches the most silent failures in complex domains.

The Relationship Test Matrix

For every foreign key relationship in your domain, create an explicit test case. The legal services domain has these key relationships to test:

Test each relationship by generating a document that exercises it and confirming the related data appears correctly. Do not assume a relationship works because the foreign key is defined — test it with actual document generation.

Testing Multi-Hop Relationships

Some of your templates require data that travels through two or more foreign key relationships to reach its destination. These are the most fragile joins and deserve the most careful testing.

In the legal services domain, the Hearing Preparation Sheet requires a three-hop path to get client information: hearings → matters → clients. Test this explicitly:

1. Select a hearing record for the Hearing Preparation Sheet template
2. Generate the document
3. Confirm that the client's name and contact information — which lives in the clients table, not in hearings or even matters — appears correctly in the client section of the generated document

If the client information is blank, the join is failing somewhere in the three-hop path. Isolate the failure by generating a simpler document that uses only the matters → clients relationship and confirming it works there first. If that works, the problem is specific to the hearings → matters join.

The Null Foreign Key Test

Some foreign keys in your domain may be intentionally nullable — the InvoiceID field in billing_entries.csv is a good example. Entries that have not yet been invoiced have no InvoiceID.

Test that templates handle nullable foreign keys gracefully. Generate an invoice that includes unbilled entries (entries with no InvoiceID) alongside billed entries. The template should either skip the unbilled entries or display them in a separate section — it should never fail or crash because a foreign key is null.

Phase 4: Document Generation Testing

Phase 4 is the core testing phase. You generate every one of your 20 templates and review every generated document for correctness, completeness, and professional quality.

The Generation Test Protocol

For each of the 20 templates, follow this four-step protocol:

Step 1: Select a primary record and generate Choose the primary record that exercises the most relationships and the most data. For a matter-based template, choose a matter that has hearings, deadlines, billing entries, and a complete client and attorney record. Generate the document.

Step 2: Verify placeholder replacement Scan the entire generated document for any text that looks like a placeholder that was not replaced. Search for << in the document. If any angle-bracket placeholders appear in the output, a field reference is incorrect. Note the field name and trace it back to the column name in the CSV.

Step 3: Verify data accuracy For each piece of data in the document, mentally trace it back to its source record. The client name — is it the correct client for this matter? The attorney — is it the lead attorney assigned to this matter, not a different attorney? The invoice amounts — do they match the records in your billing_entries.csv?

This step requires you to have your CSV files open alongside the generated document. It is tedious. It is the most important step in the entire testing process.

Step 4: Verify visual quality Read the document as a recipient would. Does it look professional? Are the headings correctly styled? Do the tables align? Is there awkward spacing between sections? Does the letterhead appear correctly? Is the page number in the footer?

A document that is data-correct but visually unprofessional reflects poorly on the domain creator and on the organizations that use it. Both dimensions matter.

The 20-Template Generation Log

Create a simple log as you work through the templates. A spreadsheet or even a text file works. Record for each template:

Template 01 — Engagement Letter
  Primary record tested: HW-2024-0047 (Thornton)
  Placeholder replacement: PASS
  Data accuracy: PASS
  Visual quality: PASS
  Issues found: None
  Status: ✓ READY

Template 03 — Invoice (Hourly)
  Primary record tested: BC-INV-2024-0018 (Delacroix)
  Placeholder replacement: PASS
  Data accuracy: FAIL — billing entry hours show decimal format (3.5)
    but client expects format "3 hours 30 minutes"
  Visual quality: PASS
  Issues found: Hour format — update billing_entries.csv or add format
    function to template
  Status: ✗ NEEDS FIX

This log becomes your pre-publication checklist. Every template must show "READY" before you publish. Every "NEEDS FIX" entry is tracked until it is resolved and retested.

Testing Every Conditional Branch

Templates with conditional sections require multiple test passes — one for each branch the conditional can take.

The Invoice template has three billing type branches: Hourly, Flat Fee, and Contingency. Testing with one Hourly matter tells you the Hourly branch works. It tells you nothing about Flat Fee or Contingency. You must test all three.

For each conditional in each template, identify every branch and test each one explicitly:

Invoice — BillingType branches: - Test 1: Generate with a Flat Fee matter (BC-2024-0031) → verify flat fee section renders, hourly time entry section does not - Test 2: Generate with an Hourly matter (HW-2024-0047) → verify time entry table renders with line items, flat fee section does not - Test 3: Generate with a Contingency matter → verify contingency notice renders, neither hourly nor flat fee sections appear

Engagement Letter — ClientType branches: - Test 1: Generate for an Individual client (Robert Thornton) → verify "Robert Thornton" appears as addressee - Test 2: Generate for an Entity client (Cascade Industries LLC) → verify company name appears with "Attn: Legal Department"

Deadline Calendar — Priority filter branches: - Test 1: Use a matter with Critical deadlines → verify Critical section appears with the correct deadlines highlighted - Test 2: Use a matter with only High and Medium deadlines → verify Critical section is either empty or absent, not broken - Test 3: Use a matter with no deadlines → verify graceful empty state, not an error

Track every conditional branch test in your generation log. A conditional that has not been tested against every branch is a conditional that may fail in production.

Phase 5: Edge Case Testing

Edge cases are the scenarios that do not appear in your happy-path sample data. They are the scenarios that real organizations will encounter within the first week of using your domain, and that will generate support requests if your templates do not handle them gracefully.

The Edge Case Inventory

For the legal services domain, these are the mandatory edge cases to test:

Empty related tables What happens when a matter has no hearings? Generate the Case Summary Letter and the Hearing Preparation Sheet for a matter with no associated hearing records. The loop should produce an empty section or a "No hearings scheduled" message — not an error.

Single-record tables What happens when the firm has only one attorney? Generate the Attorney Workload Summary. The loop should produce one row, not crash because the result set has only one record.

Maximum data scenarios What happens when a matter has 15 deadlines? Generate the Deadline Calendar. The priority sections should expand to accommodate all 15 records. The table should not overflow the page or truncate data.

Special characters in data Test data with apostrophes (O'Brien), ampersands (Smith & Associates), and quotation marks in notes fields. These characters can cause XML parsing failures in certain Word document generation scenarios. A field value of "Review O'Brien's deposition transcript" in a notes field should render correctly, not escape as ' or fail silently.

Empty optional fields Test templates with records where every optional field is empty: no notes, no case number, no opposing counsel, no preparation notes. Every section that conditionally renders based on an optional field should simply not appear — cleanly and without leaving blank space or broken formatting.

Maximum field length Matter descriptions, attorney bios, and client notes can be long. Test with the longest realistic values: a 400-character matter description, a 500-character attorney bio. Confirm that text wraps correctly within tables and does not overflow column boundaries.

Numeric edge cases An invoice with a zero balance. Billing entries with 0.1 hours. A retainer amount of zero. A flat fee of $0 for a pro bono matter. These values should render correctly and not produce division-by-zero errors or formatting failures.

Building Your Edge Case Test Data

Add edge case records to your CSV files specifically for testing. Use negative IDs or a clearly marked naming convention so they are distinguishable from your realistic sample data:

# In matters.csv — add edge case records
99,1,8,1,1,HW-EDGE-001,Matter with no hearings or deadlines,Active,2024-01-01,,5000,Flat Fee,,5000,,,,
98,1,9,1,1,HW-EDGE-002,"Matter with O'Brien & Associates as opposing counsel — special chars",Active,2024-01-01,,5000,Hourly,650,,10000,,,"O'Brien, Smith & Jones LLP","Notes: Review all filings re: O'Brien's testimony"

Run your full generation test protocol against these edge case records. When they all pass, remove them from your CSV files or keep them clearly labeled as test records — your published domain should contain realistic sample data, not test scaffolding.

The Full Validation Checklist

Before publishing, work through this complete checklist. Every item must be confirmed.

Infrastructure

[ ] Domain clones successfully from Domain Template Catalog without errors
[ ] All 10 tables appear in Data Sets with correct record counts
[ ] Domain config validation passes (20 templates, 10 tables)
[ ] All template filenames in config match actual files in word-templates/
[ ] TEMPLATE_INDEX.json is present and lists all 20 templates

Template Quality

[ ] All 20 templates appear in the Word add-in under correct categories
[ ] All 20 template descriptions are accurate and readable
[ ] All 20 primary table selectors show correct records
[ ] All 20 templates generate without errors
[ ] All 20 generated documents contain no unreplaced placeholders
[ ] All 20 generated documents are data-accurate (verified against CSV)
[ ] All 20 generated documents are visually professional

Data Relationships

[ ] Every foreign key relationship tested with explicit document generation
[ ] Every multi-hop relationship tested and verified
[ ] Nullable foreign key handling tested and graceful
[ ] No silent failures where related data is blank in generated output

Conditional Logic

[ ] Every conditional branch in every template tested explicitly
[ ] BillingType conditionals tested for all three values
[ ] ClientType conditionals tested for Individual and Entity
[ ] Priority filter conditionals tested for all priority levels
[ ] All optional field conditionals tested with field present and absent

Edge Cases

[ ] Empty related tables — loops produce clean empty states, not errors
[ ] Single-record result sets — loops handle one record correctly
[ ] Maximum realistic data volumes — no overflow or truncation
[ ] Special characters (apostrophes, ampersands) — render correctly
[ ] Empty optional fields — no blank space or broken formatting
[ ] Numeric edge cases — zero values, small decimals, large amounts

Documentation

[ ] README Quick Start verified end-to-end by someone unfamiliar with domain
[ ] All 20 template descriptions in README match actual template behavior
[ ] Troubleshooting section tested against actual induced failures
[ ] Schema reference accurate for all 10 tables
[ ] Data sources integration guide references correct connection options

When Tests Fail

Tests will fail. That is not a sign that something is wrong with your process — it is the process working as intended. Every failure caught in testing is a failure that does not reach a user.

The Fix-Regenerate-Retest Cycle

When a template test fails, the fix cycle is always the same:

If the problem is in the data: Fix the CSV file. Reload data in the add-in. Retest.

If the problem is in the configuration: Fix domain-config.json. Restart the server. Retest.

If the problem is in the template: Fix the generator script function. Regenerate the affected template (or all 20 for safety). Retest.

Never fix a generated .docx file directly. Fix the source, regenerate, retest. This discipline ensures that your fix is permanent and repeatable, not a one-time patch that disappears the next time you regenerate.

Tracking Fixes

For every failure you find and fix, add an entry to your generation log:

Template 03 — Invoice (Hourly)
  Issue found: Hours displayed as decimal (3.5) — client prefers "3.5 hours" format
  Fix applied: Updated billing_entries.csv to use "3.5 hours" in a notes
    field; updated template to display EntryHours with "hours" suffix
  Retested: PASS
  Status: ✓ READY

This log becomes your domain's quality history. When you release version 1.1 with improvements, you know exactly what was fixed in 1.0 and why.

The Beta User: Your Most Valuable Testing Tool

All five testing phases can be done by you alone. They should be. But there is one additional testing method that catches problems no solo testing process can find: giving the domain to a real member of your target audience and watching them use it.

Find one person who works in the industry your domain serves. A paralegal for the legal services domain. A homeschool co-op coordinator for a co-op domain. Give them the domain with no instructions beyond the README. Watch them try to load it, generate their first document, and navigate the template catalog.

Do not help them. Watch where they pause. Watch where they re-read the README. Watch where they click the wrong thing. Every moment of confusion is a signal that the README, the template catalog, or the Quick Start needs clarification.

A domain that passes all five testing phases but confuses its first real user is not ready. A domain that a real user can operate successfully from the README alone — that is a domain ready for the marketplace.

This test takes an hour. It is the most valuable hour in the entire development process.

Versioning and Release Preparation

When every item on the validation checklist is confirmed and your beta user has successfully used the domain, you are ready to prepare the release package.

Version Numbering

Your domain follows semantic versioning: MAJOR.MINOR.PATCH.

1.0.0 — Initial release. The first version published to the marketplace.

1.0.1 — Patch release. A bug fix that does not change any template structure or add any new templates. Fix a spelling error, correct a wrong field reference, update a CSV record.

1.1.0 — Minor release. Adds new templates, adds new data fields, or makes backward-compatible improvements to existing templates. Users who have already loaded the domain should update to get the improvements.

2.0.0 — Major release. Changes the fundamental data structure — adds or removes tables, changes primary key conventions, renames existing fields. Requires existing users to rebuild their data files. Major releases should be rare and well-communicated.

Release Package Contents

Your release package is simply the domain folder with everything in it:

legal-services/
├── domain-config.json       ← version 1.0.0
├── README.md
├── csv-data/
│   ├── firms.csv
│   ├── practice_areas.csv
│   ├── clients.csv
│   ├── attorneys.csv
│   ├── matters.csv
│   ├── hearings.csv
│   ├── deadlines.csv
│   ├── billing_entries.csv
│   ├── invoices.csv
│   └── documents_filed.csv
├── word-templates/
│   ├── Template-01-EngagementLetter.docx
│   ├── Template-01-EngagementLetter.json
│   └── ... (all 20 pairs plus TEMPLATE_INDEX.json)
├── docs/
│   ├── SCHEMA_REFERENCE.md
│   └── DATA_SOURCES_INTEGRATION.md
└── scripts/
    └── generate-legal-services-templates.js

Include the generator script. Always. It is what makes your domain forkable, trustworthy, and open. A domain without its generator script is a black box. A domain with its generator script is an invitation.

Compress the folder to a .zip file for marketplace submission. The submission process — covered in Chapter 8 — validates the package structure before it goes live.

Chapter Summary

Testing a domain has five phases: domain loading validation confirms the platform can clone and load your domain package without errors; template loading verification confirms every template appears and previews correctly; data relationship testing confirms every foreign key join produces correct output; document generation testing confirms every template generates correctly and accurately for every data scenario; and edge case testing confirms your templates handle the unexpected gracefully.

The validation checklist — infrastructure, template quality, data relationships, conditional logic, edge cases, documentation — is the complete pass/fail standard for a production-ready domain. Every item must be confirmed.

When tests fail, fix the source, regenerate, retest. Never patch a generated file directly.

A beta user from your target industry is the most valuable testing tool available. A domain that a real user can operate successfully from the README alone is a domain ready for the marketplace.

What's Next

Chapter 8 covers packaging and marketplace submission — the final step that takes your validated, production-ready domain from your local development environment to the Data Publisher marketplace where the people who need it can find it, evaluate it, and use it.

You have built something that works. Chapter 8 shows you how to share it with the world.

Volume 5 — Building Intelligent Systems: Building Organizational Knowledge Systems on Data Publisher for Word Part of the Building Intelligent Systems Series