Volume 3: Human-System Collaboration

Pattern 6: Domain-Aware Validation

Part II: Interaction Patterns - Intelligence Patterns


Opening Scenario: The Phone Number That Wasn't

Michael was the IT director at a regional hospital network. They'd recently upgraded their patient intake system, and he was proud of the modern, responsive forms. The validation was bulletproof - or so he thought.

One morning, the registration supervisor, Janet, came to his office frustrated.

"Michael, your form keeps rejecting valid phone numbers."

"That's impossible," Michael said. "We're using industry-standard phone validation. It checks the format perfectly."

Janet showed him on her computer. She entered: (412) 555-0123

The form accepted it with a green checkmark.

Then she entered: (800) 555-1234

Red error message: "Please enter a valid phone number."

"See?" Janet said. "That's our main appointment line. Patients call it all the time. But your form says it's invalid."

Michael looked at the validation code:

function validatePhone(phone) {
  const digits = phone.replace(/\D/g, '');
  return digits.length === 10;
}

"It's checking for exactly 10 digits," Michael said. "800 numbers are 10 digits. This should work."

"But it doesn't," Janet said. "Try entering it yourself."

Michael tested it. The form rejected (800) 555-1234 but accepted (412) 555-1234. He dug deeper into the code and found the problem:

function validatePhone(phone) {
  const digits = phone.replace(/\D/g, '');
  if (digits.length !== 10) return false;

  // Check area code is valid
  const areaCode = parseInt(digits.substring(0, 3));
  if (areaCode < 200) return false;  // ← Here's the problem

  return true;
}

The validation rejected area codes below 200. This was technically correct for geographic area codes - but it also rejected all toll-free numbers (800, 888, 877, 866, 855, 844, 833).

"I see the issue," Michael said. "The validation is checking syntax correctly, but it doesn't understand the domain - it doesn't know about toll-free numbers."

"There's more," Janet said. She entered: (412) 555-0199

The form accepted it.

"That's a reserved test number," Janet explained. "It's not a real phone number. The phone company reserves 555-01XX for testing. But your form thinks it's valid."

Then she entered: (412) 867-5309

"This looks like a valid number," Janet said. "But it's the number from that 1980s song. People enter it as a joke. I bet you anything this patient didn't really give us Jenny's number."

Michael realized his validation was checking syntax but not semantics. It knew phone number format but not phone number meaning.


Six months later, after working with Janet and the registration team, they had rebuilt the validation with domain intelligence:

async function validatePhone(phone, context) {
  const digits = phone.replace(/\D/g, '');

  // Layer 1: Format validation
  if (digits.length !== 10 && digits.length !== 11) {
    return {
      valid: false,
      message: "Phone numbers should be 10 digits (or 11 with country code)",
      severity: 'error'
    };
  }

  // Layer 2: Domain structure validation
  const areaCode = parseInt(digits.substring(0, 3));
  const exchange = parseInt(digits.substring(3, 6));
  const subscriber = parseInt(digits.substring(6, 10));

  // Check for toll-free (valid, but note it)
  const tollFree = [800, 888, 877, 866, 855, 844, 833];
  if (tollFree.includes(areaCode)) {
    if (context.purpose === 'patient_contact') {
      return {
        valid: false,
        message: "Please provide a direct phone number where we can reach you",
        detail: "Toll-free numbers can't receive calls. We need your personal phone.",
        severity: 'error'
      };
    }
  }

  // Check for reserved test numbers (555-01XX)
  if (exchange === 555 && subscriber >= 100 && subscriber <= 199) {
    return {
      valid: false,
      message: "This appears to be a test number",
      detail: "555-01XX numbers are reserved for testing and movies. Please enter your real number.",
      severity: 'error'
    };
  }

  // Check for known fictional/joke numbers
  const fictional = ['8675309', '5551212'];
  const last7 = digits.substring(3);
  if (fictional.includes(last7)) {
    return {
      valid: false,
      message: "This number doesn't look real",
      detail: "We need a working phone number where we can reach you for appointments and results.",
      severity: 'warning'
    };
  }

  // Layer 3: Business rule validation
  if (context.purpose === 'emergency_contact') {
    // Can't be the same as patient's own number
    if (digits === context.patientPhone) {
      return {
        valid: false,
        message: "Emergency contact can't be your own number",
        detail: "Please provide a different person we can call in an emergency.",
        severity: 'error'
      };
    }
  }

  // Layer 4: Real-world verification (optional)
  if (context.verify) {
    const lineType = await lookupLineType(digits);

    if (lineType === 'landline' && context.preferMobile) {
      return {
        valid: true,
        message: "This appears to be a landline",
        detail: "Mobile numbers are better for appointment reminders via text. Is this your mobile?",
        severity: 'info',
        suggestion: 'Consider adding mobile number'
      };
    }

    if (lineType === 'disconnected') {
      return {
        valid: false,
        message: "This number appears to be disconnected",
        detail: "Please verify the number and try again.",
        severity: 'error'
      };
    }
  }

  // All checks passed
  return {
    valid: true,
    message: "Phone number looks good",
    metadata: {
      areaCode: areaCode,
      type: tollFree.includes(areaCode) ? 'toll-free' : 'geographic',
      formatted: `(${digits.substring(0,3)}) ${digits.substring(3,6)}-${digits.substring(6)}`
    }
  };
}

Now the form understood: - Format (syntax): Is it structured like a phone number? - Domain structure: Toll-free vs geographic, reserved vs available - Business context: Emergency contacts must be different people - Real-world constraints: Is this number actually in service?

Patient registration errors dropped by 73%. Follow-up contact success rate improved by 41%. And Janet stopped coming to Michael's office with complaints.

Context

Domain-Aware Validation applies when:

Syntax is insufficient: Format checking alone doesn't ensure valid business data

Domain rules exist: The field has business logic beyond data type (phone numbers, tax IDs, account numbers, addresses)

Context matters: Validity depends on purpose (emergency contact vs appointment reminder vs billing)

User expertise varies: Some users know domain rules, others don't

Invalid data has costs: Bad data causes operational problems (failed calls, returned mail, rejected payments)

Teachable moments exist: Validation can educate users about domain constraints

Business logic changes: Rules evolve (new area codes, changed regulations, updated standards)

Problem Statement

Most validation checks syntax but ignores semantics. Common problems:

Checking format only:

// Bad: Syntax only
if (/^\d{3}-\d{2}-\d{4}$/.test(ssn)) return true;

// Accepts: 000-00-0000 (invalid SSN)
// Accepts: 666-00-0000 (never issued)
// Accepts: 900-00-0000 (not assigned)

Ignoring business rules:

// Bad: No domain knowledge
function validateDate(date) {
  return !isNaN(Date.parse(date));
}

// Accepts: birth date in the future
// Accepts: hire date before birth date
// Accepts: expiration date in the past

Missing context:

// Bad: Same validation regardless of use
function validateEmail(email) {
  return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);
}

// Accepts: personal@gmail.com for corporate email field
// Accepts: disposable@tempmail.com for account recovery
// Accepts: obvious_fake@fake.com for legitimate signup

No explanation:

// Bad: Cryptic rejection
if (!isValidCreditCard(card)) {
  return "Invalid card number";
}

// User doesn't know: Wrong format? Wrong type? Expired? Failed Luhn check?

Binary pass/fail:

// Bad: No severity levels
function validateAge(age) {
  return age >= 18 && age <= 120;
}

// Age 17: Hard error (might be valid with parental consent)
// Age 125: Hard error (clearly a typo, probably meant 25)
// No differentiation between "close" and "obviously wrong"

We need validation that understands the domain, applies business rules, considers context, and helps users understand constraints.

Forces

Accuracy vs Usability

  • Strict validation prevents bad data
  • But overly strict validation rejects valid edge cases
  • Need balance between data quality and user frustration

Static Rules vs Dynamic Context

  • Some rules are universal (SSN format)
  • Others depend on context (preferred contact method)
  • Hard-coded validation can't handle all cases

Performance vs Verification

  • Real-time external validation is slow
  • Cached/static validation is fast but may be stale
  • Users expect instant feedback

Teaching vs Blocking

  • Validation can educate users about constraints
  • But shouldn't feel like a lecture
  • Balance helpfulness with efficiency

Privacy vs Verification

  • Some validation requires external lookups
  • But external calls may leak sensitive data
  • Balance verification value with privacy cost

Solution

Implement multi-layered validation that progresses from syntax checking to domain rules to business logic to real-world verification, with context-aware messaging and severity levels.

The pattern has five validation layers:

Layer 1: Format Validation (Syntax)

Check that input matches the expected format:

function validateFormat(value, fieldType) {
  const patterns = {
    email: /^[^\s@]+@[^\s@]+\.[^\s@]+$/,
    phone: /^\d{10}$/,
    ssn: /^\d{3}-\d{2}-\d{4}$/,
    zipcode: /^\d{5}(-\d{4})?$/,
    url: /^https?:\/\/.+/
  };

  if (!patterns[fieldType]) return { valid: true };

  return {
    valid: patterns[fieldType].test(value),
    layer: 'format',
    message: valid ? null : `Please enter a valid ${fieldType} format`
  };
}

This catches typos and obviously wrong input, but doesn't verify domain semantics.

Layer 2: Domain Structure Validation

Check domain-specific rules:

function validateDomainStructure(value, domain) {
  switch(domain) {
    case 'ssn':
      return validateSSNStructure(value);
    case 'creditCard':
      return validateCreditCardStructure(value);
    case 'email':
      return validateEmailDomain(value);
    // etc.
  }
}

function validateSSNStructure(ssn) {
  const parts = ssn.split('-');
  const area = parseInt(parts[0]);
  const group = parseInt(parts[1]);

  // Area number can't be 000, 666, or 900-999
  if (area === 0 || area === 666 || area >= 900) {
    return {
      valid: false,
      layer: 'domain',
      message: "This SSN area number is not valid",
      detail: "SSNs never start with 000, 666, or 900-999"
    };
  }

  // Group number can't be 00
  if (group === 0) {
    return {
      valid: false,
      layer: 'domain',
      message: "This SSN group number is not valid",
      detail: "The middle two digits can't be 00"
    };
  }

  return { valid: true, layer: 'domain' };
}

function validateCreditCardStructure(cardNumber) {
  const digits = cardNumber.replace(/\D/g, '');

  // Luhn algorithm
  let sum = 0;
  let isEven = false;

  for (let i = digits.length - 1; i >= 0; i--) {
    let digit = parseInt(digits[i]);

    if (isEven) {
      digit *= 2;
      if (digit > 9) digit -= 9;
    }

    sum += digit;
    isEven = !isEven;
  }

  const validChecksum = (sum % 10 === 0);

  if (!validChecksum) {
    return {
      valid: false,
      layer: 'domain',
      message: "Card number appears to be invalid",
      detail: "Please check you entered it correctly"
    };
  }

  // Detect card type
  const cardType = detectCardType(digits);

  return { 
    valid: true, 
    layer: 'domain',
    metadata: { cardType, lastFour: digits.slice(-4) }
  };
}

Layer 3: Business Rule Validation

Check organization-specific constraints:

function validateBusinessRules(value, field, context) {
  const rules = {
    birthDate: validateBirthDateRules,
    hireDate: validateHireDateRules,
    emergencyContact: validateEmergencyContactRules,
    // etc.
  };

  if (rules[field]) {
    return rules[field](value, context);
  }

  return { valid: true, layer: 'business' };
}

function validateBirthDateRules(birthDate, context) {
  const date = new Date(birthDate);
  const now = new Date();
  const age = (now - date) / (365.25 * 24 * 60 * 60 * 1000);

  // Future date
  if (date > now) {
    return {
      valid: false,
      layer: 'business',
      message: "Birth date can't be in the future",
      severity: 'error'
    };
  }

  // Unreasonably old
  if (age > 120) {
    return {
      valid: false,
      layer: 'business',
      message: "This birth date seems unlikely",
      detail: "Did you mean " + (now.getFullYear() - parseInt(birthDate.substring(2))) + "?",
      severity: 'error'
    };
  }

  // Minor (warning, not error)
  if (age < 18 && context.requiresAdult) {
    return {
      valid: true,
      layer: 'business',
      message: "Applicant appears to be under 18",
      detail: "Parental consent may be required",
      severity: 'warning'
    };
  }

  return { valid: true, layer: 'business' };
}

function validateEmergencyContactRules(contact, context) {
  // Can't be same as patient
  if (contact.phone === context.patient.phone) {
    return {
      valid: false,
      layer: 'business',
      message: "Emergency contact must be a different person",
      detail: "Please provide someone else we can reach if you're unavailable",
      severity: 'error'
    };
  }

  // Warn if in different time zone
  if (contact.timezone !== context.patient.timezone) {
    return {
      valid: true,
      layer: 'business',
      message: "Emergency contact is in a different time zone",
      detail: `${contact.name} is in ${contact.timezone} (${getTimeDifference()} hours difference)`,
      severity: 'info'
    };
  }

  return { valid: true, layer: 'business' };
}

Layer 4: Real-World Verification

Validate against external systems:

async function validateRealWorld(value, verificationType) {
  switch(verificationType) {
    case 'address':
      return await validateAddressUSPS(value);
    case 'taxId':
      return await validateTaxIdIRS(value);
    case 'email':
      return await validateEmailMX(value);
    case 'phone':
      return await validatePhoneCarrier(value);
  }
}

async function validateAddressUSPS(address) {
  try {
    const response = await fetch('/api/usps/verify', {
      method: 'POST',
      body: JSON.stringify(address)
    });

    const result = await response.json();

    if (result.verified) {
      return {
        valid: true,
        layer: 'realworld',
        message: "Address verified",
        metadata: {
          standardized: result.standardizedAddress,
          dpv: result.deliveryPointValidated
        }
      };
    } else if (result.suggestions.length > 0) {
      return {
        valid: true,
        layer: 'realworld',
        message: "Did you mean this address?",
        suggestions: result.suggestions,
        severity: 'warning'
      };
    } else {
      return {
        valid: false,
        layer: 'realworld',
        message: "Address not found in USPS database",
        detail: "Please verify the address is correct",
        severity: 'error'
      };
    }
  } catch (error) {
    // External service failed - don't block user
    return {
      valid: true,
      layer: 'realworld',
      message: "Could not verify address (service unavailable)",
      severity: 'info',
      skipValidation: true
    };
  }
}

Layer 5: Contextual Intelligence

Adapt validation based on usage context:

function getValidationContext(field, formContext) {
  return {
    purpose: formContext.purpose,  // 'patient_contact', 'billing', 'emergency'
    userRole: formContext.userRole,  // 'patient', 'staff', 'admin'
    requiredLevel: getRequiredLevel(field, formContext),
    preferredFormat: getUserPreferences(field, formContext.userId),
    historicalData: getHistoricalPatterns(field, formContext)
  };
}

async function validateWithContext(value, field, formContext) {
  const context = getValidationContext(field, formContext);
  let results = [];

  // Layer 1: Format
  results.push(await validateFormat(value, field.type));

  // Layer 2: Domain
  if (results.every(r => r.valid)) {
    results.push(await validateDomainStructure(value, field.domain));
  }

  // Layer 3: Business rules
  if (results.every(r => r.valid)) {
    results.push(await validateBusinessRules(value, field.name, context));
  }

  // Layer 4: Real-world (only if critical field and context requires it)
  if (results.every(r => r.valid) && context.requiredLevel === 'verified') {
    results.push(await validateRealWorld(value, field.verificationType));
  }

  // Combine results
  return combineValidationResults(results, context);
}

function combineValidationResults(results, context) {
  // Find highest severity error
  const errors = results.filter(r => !r.valid);
  if (errors.length > 0) {
    return errors.sort((a, b) => 
      getSeverityLevel(b.severity) - getSeverityLevel(a.severity)
    )[0];
  }

  // Find warnings/info
  const warnings = results.filter(r => r.severity === 'warning' || r.severity === 'info');
  if (warnings.length > 0) {
    return warnings[0];  // Show first warning
  }

  // All valid
  return {
    valid: true,
    message: context.showSuccessMessage ? "Looks good!" : null,
    metadata: results.reduce((acc, r) => ({...acc, ...r.metadata}), {})
  };
}

Implementation Details

See complete implementation code in the full pattern above (DomainAwareValidator class with built-in validators for email, SSN, phone, credit card, and extensible framework).

Consequences

Benefits

Higher Data Quality: - Catches semantic errors, not just syntax - Prevents obviously invalid data - Teaches users about constraints

Better User Experience: - Helpful error messages - Suggestions for corrections - Graduated severity (error vs warning vs info)

Reduced Support Burden: - Fewer "why was my input rejected?" questions - Self-service correction - Educational validation

Business Intelligence: - Track validation failures to find UX problems - Identify common mistakes - Discover data quality issues

Maintainable Rules: - Domain logic centralized - Business rules separate from format checks - Easy to update as rules change

Liabilities

Implementation Complexity: - More sophisticated than regex validation - Requires domain knowledge - Multiple validation layers to maintain

Performance Concerns: - External validation is slow - May need caching strategies - Can't always validate synchronously

False Positives: - Overly strict validation rejects edge cases - Users may know their data is valid - Need override mechanisms

External Dependencies: - Real-world validation requires third-party services - Services may be unavailable - Privacy implications of external calls

Maintenance Burden: - Domain rules change (new area codes, updated regulations) - Requires ongoing updates - Need monitoring for rule effectiveness

Domain Examples

See full pattern above for detailed examples in: - Healthcare (patient phone validation) - Financial Services (EIN validation) - Real Estate (property address validation) - Legal Services (case number validation)

Prerequisites: - Volume 3, Pattern 5: Error as Collaboration (validation errors need good messaging)

Synergies: - Volume 3, Pattern 7: Calculated Dependencies (validated fields inform calculations) - Volume 3, Pattern 10: Semantic Suggestions (validation suggests corrections) - Volume 3, Pattern 14: Cross-Field Validation (domain rules often span fields) - Volume 3, Pattern 24: Real-Time External Validation (external APIs for verification)

Conflicts: - Offline-first designs (can't always do real-world validation) - Performance-critical forms (external validation adds latency)

Alternatives: - Post-submission validation (if real-time isn't feasible) - Batch validation (for bulk imports) - Progressive validation (validate more as user demonstrates expertise)

Known Uses

Stripe Payment Forms: Multi-layer card validation (format → Luhn → BIN lookup → fraud detection)

TurboTax: Domain-aware validation for tax forms (SSN structure, income ranges, deduction limits)

Healthcare Portals (Epic, Cerner): Phone/address validation with USPS verification

Banking Applications: Account number validation with check digits and institutional routing

E-commerce Checkout (Amazon, Shopify): Address validation with carrier verification

Government Forms (IRS, USCIS): Strict domain validation for tax IDs, case numbers, application codes

Real Estate Platforms (Zillow, Redfin): Property address validation with MLS cross-reference


Further Reading

Academic Foundations

Domain Modeling: - Evans, E. (2003). Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley. - Ubiquitous language and domain entities—foundation for domain-aware validation - https://www.domainlanguage.com/ddd/ - Vernon, V. (2013). Implementing Domain-Driven Design. Addison-Wesley. - Aggregates, value objects, and domain validation rules

Semantic Validation: - Berners-Lee, T., Hendler, J., & Lassila, O. (2001). "The Semantic Web." Scientific American, 284(5), 34-43. - Semantic understanding of data for intelligent validation - https://doi.org/10.1038/scientificamerican0501-34 - Fielding, R. T. (2000). "Architectural Styles and the Design of Network-based Software Architectures." Doctoral dissertation, UC Irvine. - REST constraints including uniform interface and hypermedia - Foundation for API-based domain validation

Data Quality: - Redman, T. C. (2001). Data Quality: The Field Guide. Digital Press. - Accuracy, completeness, consistency in data validation - Batini, C., & Scannapieco, M. (2016). Data and Information Quality. Springer. - Dimensions of data quality and validation strategies - https://doi.org/10.1007/978-3-319-24106-7

Implementation Guides

Validation Libraries: - Validator.js: https://github.com/validatorjs/validator.js - String validators for email, URL, credit card, ISBN, etc. - Zod (TypeScript): https://zod.dev/ - Schema validation with TypeScript type inference - Joi: https://joi.dev/ - Schema validation with domain-specific rules - Ajv (JSON Schema): https://ajv.js.org/ - JSON Schema validator for complex domain models

Domain-Specific Validators: - Credit Card Validator: https://github.com/braintree/card-validator - Luhn algorithm, card type detection, expiration validation - Phone Number Validation: libphonenumber - https://github.com/google/libphonenumber - Google's international phone number library - Address Validation: Smarty (formerly SmartyStreets) - https://www.smarty.com/products/us-address-verification - USPS-certified address validation - Email Validation: https://github.com/mailcheck/mailcheck - Domain suggestions and MX record checking

Regular Expressions: - RegExr: https://regexr.com/ - Interactive regex tester and reference - Regex101: https://regex101.com/ - Regex debugger with explanation - OWASP Validation Regex Repository - https://owasp.org/www-community/OWASP_Validation_Regex_Repository

Volume 1 Foundations: - Chapter 3: "Document Ontology - A Formal Framework" - Domain-specific data structures - Chapter 8: "Architecture of Domain-Specific Systems" - Validation architecture - Chapter 10: "Domain Knowledge Acquisition" - Capturing domain rules

Volume 2 Patterns: - Volume 2, Pattern 4: "Intelligent Formatting" - Auto-formatting domain-specific inputs - Volume 2, Pattern 9: "Lookup Validation" - Real-time API validation - Pattern 17: "Fuzzy Matching" - Accepting variations in domain data

Volume 3 Integration: - Volume 3, Pattern 5: "Error as Collaboration" - Providing helpful domain-specific error messages - Volume 3, Pattern 8: "External Data Integration" - Validating against third-party domain data - Pattern 16: "Blockchain Verification" - Immutable domain validation records

Industry Examples

Financial Validation: - Stripe: https://stripe.com/docs/js/elements_object/create_element - Credit card validation with card type detection - Plaid: https://plaid.com/ - Bank account and routing number validation - PayPal: https://developer.paypal.com/ - Payment method validation

Healthcare Validation: - Epic Systems: ICD-10 diagnosis code validation - Cerner: NDC drug code validation - HIPAA EDI Validators: https://www.hipaaspace.com/ - Healthcare transaction validation (837, 835, etc.)

Government Systems: - IRS e-File: https://www.irs.gov/e-file-providers/validating-schemas - Tax return schema validation - USCIS: Immigration form validation - Case number, A-number, receipt number validation - SSA: https://www.ssa.gov/ - Social Security Number verification (limited API access)

Standards and Compliance

Data Standards: - ISO 8601: Date and time formats - https://www.iso.org/iso-8601-date-and-time-format.html - ISO 3166: Country codes (alpha-2, alpha-3, numeric) - https://www.iso.org/iso-3166-country-codes.html - ISO 4217: Currency codes - https://www.iso.org/iso-4217-currency-codes.html

Industry-Specific Standards: - ACORD: Insurance industry data standards - https://www.acord.org/standards-architecture/acord-data-standards - HL7 FHIR: Healthcare data exchange - https://www.hl7.org/fhir/ - MISMO: Mortgage industry standards - https://www.mismo.org/

Validation Standards: - JSON Schema: https://json-schema.org/ - Vocabulary for annotating and validating JSON documents - XML Schema (XSD): https://www.w3.org/XML/Schema - W3C recommendation for XML validation - OpenAPI Specification: https://swagger.io/specification/ - API request/response validation

Research and Tools

Address Validation: - Google Maps Geocoding API: https://developers.google.com/maps/documentation/geocoding - Address validation and standardization - USPS Address API: https://www.usps.com/business/web-tools-apis/ - Official USPS address validation - Melissa Data: https://www.melissa.com/ - Global address verification

Phone Number Validation: - Twilio Lookup API: https://www.twilio.com/docs/lookup/api - Phone number validation and carrier lookup - Numverify: https://numverify.com/ - Phone number validation API - Phone Number Validator (libphonenumber): https://github.com/google/libphonenumber - Google's international phone number library

Tax ID Validation: - TIN Matching (IRS): https://www.irs.gov/tax-professionals/tin-matching - Validate SSN/EIN for payee reporting - VAT Validation (EU): https://ec.europa.eu/taxation_customs/vies/ - VIES VAT number validation

Industry Validators: - VIN Decoder: https://vpic.nhtsa.dot.gov/api/ - Vehicle identification number validation - ISBN Validator: https://isbntools.readthedocs.io/ - Book identification number validation - IATA Codes: https://www.iata.org/ - Airport and airline code validation