Skip to main content

Documentation Generation

Automatically generate comprehensive documentation for your database tables and columns using AI, ensuring consistent metadata across your data catalog.

VIDEO TUTORIAL⏱️ 3 mins

📹 Auto-Documentation with AI

Learn how the Data Documentation Agent automatically generates descriptions for your tables and columns, maintaining comprehensive data documentation.

Overview​

The Data Documentation Agent automatically analyzes your database schema and generates business-friendly descriptions for tables and columns. This ensures your data catalog has comprehensive, consistent documentation without manual effort.

How It Works​

Automatic Documentation Process​

  1. Associate the Data Documentation Agent with your data source
  2. After catalog scanning, the agent analyzes table and column structures
  3. AI generates contextual descriptions based on:
    • Column names and data types
    • Table relationships
    • Data patterns and constraints
    • Business context clues

Documentation Coverage​

The agent documents:

  • Table Descriptions: Purpose and business context of each table
  • Column Descriptions: What each field represents and its usage
  • Confidence Scoring: Quality assessment of generated documentation
  • Review Status: Tracks which documentation needs human review

Viewing Documentation​

1. Dashboard Widget​

The documentation widget on the data source dashboard shows:

  • Documentation Coverage: Percentage of documented vs undocumented entities
  • Tables Documented: Count of tables with descriptions
  • Columns Documented: Count of columns with descriptions
  • Review Status: Items needing review

2. Catalog Integration​

Documentation appears throughout the catalog:

  • Table Level: Hover over table names to see descriptions
  • Column Level: Descriptions visible in the column details view
  • Inline Display: Documentation integrated into the browsing experience

3. Agent View​

Access the dedicated Data Documentation Agent view for detailed management:

Activities View​

The main activities view displays all generated documentation:

FieldDescription
Entity NameTable or column name
Entity TypeTable or Column indicator
DescriptionGenerated documentation text
Confidence ScoreAI confidence (0-100%)
Quality StatusBased on confidence score thresholds
TimestampWhen documentation was generated

Filtering and Sorting​

Filter Options​

Available filters in the activities view:

  • Confidence dropdown: All Levels, High (80%+), Medium (50-79%), Low (<50%)
  • Coverage dropdown: All, Documented, Needs Review, Undocumented
  • Type dropdown: All Types, Tables, Columns
  • Time dropdown: All Time, Today, Last 7 Days, Last 30 Days
  • Table dropdown: All Tables, or specific table (if available)
  • Search box: Find specific activities
  • Clear button: Reset all filters when active

Sort Options​

Organize documentation by:

  • Date: Most recent first or oldest first
  • Confidence: Highest to lowest or lowest to highest
  • Table: Alphabetical by table name
  • Coverage: Documentation status

Identifying Items Needing Review​

Low confidence documentation typically indicates:

  • Ambiguous Names: Columns like "AV", "FFF", "AF" with unclear meaning
  • Generic Fields: Common names without business context
  • Complex Relationships: Fields requiring domain knowledge

Sort by confidence (low to high) to prioritize items needing human review.

Review Mode​

Enable Review Mode for systematic documentation validation:

Entering Review Mode​

  1. Toggle the Review Mode switch in the Documentation Activities header
  2. Interface switches to table-grouped review layout
  3. Tables can be expanded/collapsed with chevron icons
  4. Progress tracking shows total reviewed and pending items

Review Workflow​

For each documentation item:

Approve Documentation​

  • Click the Approve button (thumbs up icon)
  • Confirms the AI-generated description is accurate
  • Documentation status changes to "Approved"
  • Progress counter updates automatically

Reject and Edit​

  1. Click the Provide Feedback button (thumbs down icon)
  2. A popover opens with:
    • Improved Description text area (pre-filled with current description)
    • Notes text area (optional - for additional context)
  3. Modify the description as needed
  4. Click Save Improvement
  5. Documentation is updated with your changes

Inline Editing​

In review mode, you can also:

  • Quick Edit: Click directly on descriptions to edit inline
  • Bulk Actions: Select multiple items for batch approval
  • Table Navigation: Expand/collapse tables to review columns

Review Progress​

Track your progress with:

  • Progress Bar: Visual indicator of completion percentage
  • Counter Display: "X reviewed / Y total" items
  • Status Filters: View pending, approved, or all items

Quality Assessment​

Confidence Scores​

Documentation confidence is calculated based on:

  • Name Clarity: How descriptive the entity name is
  • Context Available: Related tables and foreign keys
  • Pattern Recognition: Similar entities in the database
  • Data Type Alignment: Consistency with data types

Quality Status Categories​

StatusConfidence RangeAction Required
High Confidence80-100%Optional review
Medium Confidence50-79%Recommended review
Low ConfidenceBelow 50%Manual review required

Best Practices​

Regular Reviews​

  • Prioritize low-confidence items for review
  • Focus on business-critical tables first
  • Schedule periodic reviews of documentation

Improving Documentation Quality​

  • Provide corrections to train the AI model
  • Add business context in improvement notes
  • Use consistent terminology across descriptions

Collaboration​

  • Share review responsibilities across team members
  • Document domain-specific terminology
  • Maintain a glossary of business terms

Integration with Other Features​

Data Catalog​

  • Documentation appears in table and column views
  • Searchable through catalog search
  • Exportable with catalog reports

Data Lineage​

  • Documentation helps understand data flow
  • Provides context for impact analysis
  • Clarifies transformation logic

Team Collaboration​

  • Share documented catalog with stakeholders
  • Maintain consistent understanding of data
  • Onboard new team members faster

API Access​

Documentation can be accessed and managed programmatically:

  • Query documentation by table or column
  • Submit documentation updates
  • Export documentation in bulk
  • Integrate with data governance tools

Troubleshooting​

Common Issues​

IssueCauseSolution
Missing documentationAgent not associatedAdd Documentation Agent to data source
Low confidence scoresUnclear naming conventionsReview and provide corrections
Documentation not updatingCache delayWait for next agent run or trigger manually
Incomplete coverageLarge databaseAgent processes in batches, wait for completion

Related Topics​