Troubleshooting & Debugging

When building synthetic scenarios, things don't always work perfectly on the first try. Modern web applications are complex, and headless browsers can behave differently than a real user.

This guide is designed for DevOps engineers and SREs to quickly diagnose and resolve common issues encountered while using the Synthetic Exporter.

1. Debugging Failed XPaths

If a scenario is failing at a fill, click, or wait_for_selector step, it is usually because the Playwright engine cannot find the element on the page.

Common Causes & Solutions

• Asynchronous Rendering (React/Vue/Angular): The element might not exist in the DOM immediately. Playwright natively waits for elements to be visible, but if the element is removed and re-added to the DOM quickly, the selector might detach. Use wait_for_selector explicitly before clicking, or rely on stable data-testid attributes.
• iFrames: The exporter currently evaluates selectors in the main frame. If the element you are targeting (like a third-party login or Stripe checkout) is inside an <iframe>, a standard XPath will not find it. (Note: Full cross-frame support is a planned feature, but currently, you should avoid targeting elements inside cross-domain iframes).
• Shadow DOM: Standard XPaths cannot pierce Shadow DOM boundaries (common in Web Components). If your app uses Shadow DOM, consider switching your selectors from xpath= to Playwright's native text or CSS selectors if supported by your application structure.
• Dynamic Classes: Avoid using XPaths that rely on auto-generated CSS classes (e.g., //div[@class='sc-bdfBwQ gBikH']). Always use stable identifiers like id, name, or data-testid.

2. Handling WAFs, Cloudflare, and CAPTCHAs

One of the most frequent reasons a synthetic journey fails in production is that your Web Application Firewall (WAF) or DDoS protection (like Cloudflare, Akamai, or AWS WAF) blocks the headless browser.

Diagnosis

If your scenario fails on the very first goto step with a timeout, or a screenshot step reveals an "Access Denied" or "Checking your browser" page, you are being blocked by a WAF. Headless Chromium broadcasts its nature via the User-Agent and webdriver browser flags.

Solutions

1. IP Allowlisting (Recommended): The most robust solution is to assign a static IP (or NAT Gateway IP) to the server/pod running the Synthetic Exporter, and explicitly allowlist that IP address in your WAF rules to bypass CAPTCHAs and bot protection.
2. Custom Headers: Configure your ingress controller or WAF to bypass bot protection if a specific, secret HTTP header is present (though the current version of Synthetic Exporter relies purely on IP or credentials).
3. Environment Isolation: Run your synthetic tests against a staging or dedicated synthetic testing environment where aggressive bot protection is disabled.

3. Interpreting error_type_name

The exporter is designed to give you precise telemetry. When a step fails, the synthetic_ui_journey_errors_total Prometheus counter is incremented, and it carries the error_type_name you defined in your config.yaml.

How to use this effectively: If your alert fires and you see error_type_name="login_button_missing", you immediately know that the page loaded, but the UI changed, or the authentication service is down and the button never rendered.

If you see error_type_name="navigation_failed", it implies a network-level issue or DNS resolution failure preventing the initial page load.

Always group your Grafana dashboard panels and Alertmanager alerts by error_type_name to accelerate root cause analysis.

4. Internal Logs & Container Debugging

If the metrics are not providing enough context, or the exporter itself is crashing, you need to read the internal logs.

The Synthetic Exporter uses structured JSON logging (via Go's slog) which is highly compatible with log aggregators like Datadog, Loki, or ELK.

Exposing Logs

• Docker: Run docker logs <container_id> to see the exact execution logs. You will see detailed logs for every step attempted, the duration of the step, and the exact error returned by Playwright.
• Kubernetes: Run kubectl logs -l app=synthetic-exporter to view the pod logs.

What to look for:

• browser launch failed: The container might be out of memory (OOM), or the underlying OS lacks dependencies (unlikely if using the official Docker image). Check your container memory limits.
• Timeout 30000ms exceeded: Playwright waited 30 seconds for your element to appear, but it never did. Review your XPath, or verify via manual testing that the page isn't genuinely broken or performing slowly.