Automating User Experience Issues: A Continuous AI Workflow for PostHog Session Analysis

Every day, your users are telling you stories through their behavior. Rage clicking on broken buttons, abandoning long forms, struggling through confusing navigation flows. These stories are often buried in raw session data, error logs, and analytics dashboards.

Traditional approaches to user experience monitoring create a bottleneck: someone needs to regularly review session recordings, identify patterns, prioritize issues, and create actionable tickets for the development team. This manual process is slow, inconsistent, and often gets deprioritized.

What if this entire workflow could run automatically in the background, surfacing the most critical UX issues while you sleep?

Our Solution: Conversational Automation

We built a Continuous AI workflow that completely replaces traditional scripting approaches with natural language prompts. Instead of maintaining bash scripts, our system uses Continue CLI to:

1. Fetch recent session recordings directly from PostHog API
2. Filter for problematic sessions (console errors, unusually long durations)
3. Analyze patterns using Continue CLI's AI capabilities
4. Create prioritized GitHub issues via GitHub API
5. Run automatically on a daily schedule via GitHub Actions

The New Paradigm: Prompts Over Scripts

Rather than maintaining bash scripts, we use structured natural language prompts and rules that Continue CLI executes directly:

Session Analysis Prompt:

# Prompt
   Perform a comprehensive UX analysis and issue tracking based on PostHog session recordings.

   ## Analysis Process
   1. Fetch PostHog session recordings using:
      - Endpoint: `{{POSTHOG_HOST}}/api/projects/{{POSTHOG_PROJECT_ID}}/session_recordings/?limit=20`
      - Required headers (with valid secrets).
   2. Analyze data with the following priority:
      - Console errors (highest priority).
      - Session duration vs. activity score ratios.
      - Bounce rates (0–1 second sessions).
      - Click/interaction patterns.
      - Page-specific retention issues.

   ## Issue Identification
   - Identify the **top 3 UX issues** based on frequency and impact.
   - Categorize issues by severity:
   - **Critical**: console errors
   - **High**: engagement-related problems
   - **Medium**: retention issues
   - Provide **evidence-based recommendations** for each identified issue.

Issue Creation Prompt:

I have analyzed PostHog session data and identified UX issues. Now I need you to create GitHub issues using the GitHub API.

Use these details from environment variables:
- GH_PAT: My GitHub Personal Access Token
- Repository: {owner}/{repo}

For each issue in my analysis:
1. Parse the issue title, body content, and priority level
2. Create a GitHub issue via API POST to: https://api.github.com/repos/{owner}/{repo}/issues
3. Set labels based on priority levels...

For the full workflow, we used the PostHog MCP and set up a GitHub action that runs daily. Once working, you'll see something like this, depending on your session recording data:

What Makes This Continuous AI

This is AI that operates autonomously within our development workflow:

• Always Working: Runs daily at 6 AM UTC without human intervention
• Contextual Intelligence: Makes API calls, processes data, and understands our specific patterns
• Actionable Output: Creates properly formatted, prioritized GitHub issues that fit our development process
• Self-Contained: Handles everything from authentication to issue creation
• Conversational Logic: Uses natural language instead of brittle scripts

The Power of Continue CLI as Infrastructure

While most developers know Continue as an editor extension, Continue CLI opens up automation possibilities far beyond code completion. In our workflow, it serves as the execution engine by handling:

• API Authentication: Securely managing PostHog and GitHub credentials
• Data Processing: Parsing JSON responses and filtering session data
• Pattern Recognition: Analyzing complex user behavior patterns
• Issue Creation: Making GitHub API calls with proper error handling
• Workflow Orchestration: Managing the entire end-to-end process

Real-World Impact: From Data to Decisions

Instead of hoping someone notices UX issues buried in analytics dashboards, your development team receives a prioritized list of UX issues discovered overnight. Not generic analytics reports, but specific, actionable problems with clear reproduction steps and fix recommendations. Issues that real users encountered, analyzed for patterns, and translated into development-ready tickets.

This is Continuous AI handling the entire workflow automatically. The system fetches session data, identifies problematic patterns, analyzes root causes, and creates GitHub issues that fit seamlessly into existing development processes.

Instead of asking AI to help when you remember to, Continuous AI works constantly in the background, handling routine tasks and surfacing insights without human intervention.

The Future of Development Intelligence

This approach points toward a future where development teams operate more like intelligence organizations. Instead of manually hunting for problems, automated systems continuously gather signals, analyze patterns, and surface actionable insights. Human developers focus on solving problems rather than finding them.

Whether you're analyzing user sessions, automating code reviews, or building custom development workflows, the pattern remains consistent: describe what you want in natural language, let Continue CLI handle the technical complexity, and iterate based on results.

This is just the beginning of what's possible when AI becomes an infrastructure layer instead of just your coding assistant.