From Test Cases to Test Intelligence: What I Learned Building a QA Knowledge Graph

Most automation frameworks answer a familiar question:

Did the test pass or fail?

But as automation suites grow, more difficult questions emerge:

• Which tests are affected by a code change?
• Which business rules have no test coverage?
• Why should a particular test be included in regression?
• Which tests are indirectly affected through component dependencies?
• Are there automated tests that no longer trace back to a documented requirement?

I have been exploring how Knowledge Graphs can provide the foundation for a QA Test Intelligence system capable of answering these questions.

1. This post documents what I learned from the first two lessons of that journey.

And suppose the application contains:

• LoginController
• AuthenticationService
• PasswordValidator
• AccountLockService

We also have automated tests:

• TC-LOGIN-001
• TC-LOGIN-002
• TC-LOGIN-003
• TC-LOGIN-004 This is where the Knowledge Graph becomes interesting.

2. Representing QA Knowledge as a Graph

Instead of treating test cases as isolated records, we can model QA information as entities and relationships.

For example:

(Login)-[:CONTAINS]->(BR-001)

(TC-LOGIN-001)-[:VERIFIES]->(BR-001)

(BR-001)-[:IMPLEMENTED_BY]->(AuthenticationService)

The entities are nodes.

Examples:

• Feature
• BusinessRule
• TestCase
• Component
• Defect

The connections are relationships, or edges.

Examples:

• CONTAINS
• VERIFIES
• IMPLEMENTED_BY
• DEPENDS_ON

This creates something much more useful than a list of test cases.

It creates traceability that can be traversed computationally.

3. My First Important Discovery: Missing Relationships Are Data

BR-004 states:

After five failed login attempts, the account becomes locked.

Initially, it was tempting to associate an existing negative-login test with this rule. For example:

TC-LOGIN-004 Verify login fails with empty credentials.

But that test doesn’t verify account locking.

Therefore creating this relationship would be incorrect:

(TC-LOGIN-004)-[:VERIFIES]->(BR-004)

The correct graph contains no such relationship.

That produces:

BR-004 <-VERIFIES - ?

And suddenly the absence of an edge tells us something:

BR-004 has no known verifying test.

That is a coverage gap.

This changed how I thought about the graph.

A missing relationship isn’t necessarily incomplete data that should immediately be filled.

Sometimes:

Missing relationship = useful QA intelligence

4. The Graph Can Detect Different Types of Gaps

Consider another test:

TC-LOGIN-004 Verify login fails with empty credentials.

Suppose none of the supplied business rules describes empty credentials.

The graph now has the opposite problem.

TC-LOGIN-004 -VERIFIES->?

This doesn’t automatically mean the test is invalid.

Instead, it creates a traceability gap.

Possible explanations include:

Missing business rule

Undocumented requirement

Obsolete test

Incorrect graph mapping

The correct response from a Test Intelligence system should therefore be:

TRACEABILITY GAP

Test: TC-LOGIN-004

Finding: No verified business-rule relationship exists.

Action: Investigate.

This distinction is important.

The graph shouldn’t invent relationships simply to make the data look complete.

5. From Traceability to Change-Impact Analysis

This is where the Knowledge Graph starts becoming particularly useful.

Suppose we introduce:

TC-LOGIN-005

which explicitly tests account locking.

Now:

TC-LOGIN-005
      |
   VERIFIES
      ↓
    BR-004
      |
IMPLEMENTED_BY
      ↓
AccountLockService

A developer changes:

AccountLockService

The graph can traverse backwards:

AccountLockService
        ↑
IMPLEMENTED_BY
        |
      BR-004
        ↑
     VERIFIES
        |
  TC-LOGIN-005

The engine can therefore report:

CHANGED COMPONENT
AccountLockService

POTENTIALLY IMPACTED TEST
TC-LOGIN-005

Notice the wording potentially impacted.

The graph doesn’t prove that TC-LOGIN-005 will fail.

Instead, it provides evidence that the test is related to behaviour implemented by the changed component.

6. One Business Rule Can Have Multiple Tests

The model becomes more useful when multiple tests verify the same behaviour.

For example:

TC-LOGIN-005 ──VERIFIES──┐
                         ↓
                       BR-004
                         ↑
TC-LOGIN-006 ──VERIFIES──┘
                         |
                  IMPLEMENTED_BY
                         ↓
                AccountLockService

If AccountLockService changes, traversal identifies:

TC-LOGIN-005
TC-LOGIN-006

Both are potentially impacted.

Another test:

TC-LOGIN-007
   |
VERIFIES
   ↓
BR-002
   |
IMPLEMENTED_BY
   ↓
AuthenticationService

would not be selected through the same AccountLockService → BR-004 traversal.

This is the beginning of intelligent regression-test selection.

Instead of saying:

“The login code changed, so run every login test.”

we can potentially say:

“These tests have graph paths connecting them to the changed component.”

That provides an explainable reason for selecting them.

7. Direct and Indirect Impact

The next concept was dependency-aware impact analysis.

Suppose:

AccountLockService
      |
 DEPENDS_ON
      ↓
AuthenticationService

Now imagine AuthenticationService changes.

There are two forms of impact.

Direct impact

TC-LOGIN-007
      |
   VERIFIES
      ↓
    BR-002
      |
IMPLEMENTED_BY
      ↓
AuthenticationService ← CHANGED

Indirect impact

Now consider:

TC-LOGIN-005
      |
   VERIFIES
      ↓
    BR-004
      |
IMPLEMENTED_BY
      ↓
AccountLockService
      |
 DEPENDS_ON
      ↓
AuthenticationService ← CHANGED

TC-LOGIN-005 is potentially impacted indirectly.

The same applies to another BR-004 test:

TC-LOGIN-006

The resulting analysis becomes:

This is significantly richer than simple tagging.

8. Why This Matters for Large Automation Suites

Imagine a CI pipeline containing:

2,500 automated tests

A component changes.

The simplest strategy is:

Code change
     ↓
Run 2,500 tests

Another strategy might use manually maintained tags:

Code change
     ↓
Determine feature
     ↓
Run tests tagged "login"

A Knowledge Graph potentially enables:

Code Change
      ↓
Changed Component
      ↓
Traverse Dependencies
      ↓
Affected Business Rules
      ↓
Tests Verifying Those Rules
      ↓
Risk-based Regression Candidates

At the same time, traversal could reveal:

Changed Component
      ↓
Affected Business Rule
      ↓
NO VERIFYING TEST
      ↓
Coverage Risk

So the same model can potentially support both:

Regression selection and coverage-gap detection.

9. A Critical Modelling Lesson

One mistake I made during the exercise was connecting entities because the relationship seemed technically plausible.

For example:

PasswordValidator -DEPENDS_ON-> AccountLockService

That relationship might exist in a real application.

But unless the architecture, source code, configuration, API interaction, or another trusted source confirms it, the Knowledge Graph shouldn’t treat it as established fact.

This leads to three principles I want the Test Intelligence system to preserve:

Unknown ≠ False

Missing ≠ Incorrect

Inferred ≠ Verified

This becomes even more important when LLMs are eventually introduced.

An LLM may be useful for proposing relationships.

It should not automatically convert every plausible relationship into graph truth.

10. Where the Test Intelligence Engine Is Heading

The graph currently looks conceptually like this:

Feature
   |
CONTAINS
   ↓
BusinessRule
   ↑          \
VERIFIES       IMPLEMENTED_BY
   |                \
TestCase           Component
                       |
                   DEPENDS_ON
                       ↓
                   Component

Even this small graph can begin answering questions such as:

Which tests verify BR-004?

Which business rules have no tests?

Which tests have no requirement traceability?

Which components implement a business rule?

Which tests are potentially affected by a component change?

Which tests are indirectly affected through dependencies?

The longer-term objective is to extend this toward:

Requirements
Business Rules
Source Code
Components
APIs
Automated Tests
Manual Tests
Defects
CI Results
Code Changes
Risk
Coverage

and eventually combine the structured graph with LLMs/RAG.

But the important lesson so far is that the intelligence shouldn’t start with the LLM.

It starts with the relationships between QA and software-engineering knowledge.

Key Takeaway My biggest takeaway from these lessons is:

Test intelligence isn’t just about executing tests. It is about understanding why a test exists, what behaviour it verifies, what implements that behaviour, and how changes propagate through those relationships.

The Knowledge Graph gives us a structure for representing that information.

The next stage of my learning will move deeper into graph traversal and change-impact analysis, before introducing databases such as Neo4j and eventually exploring how LLMs can operate on top of this structured QA knowledge.