The filter step is where most pipelines die
After auditing 12 client pipelines over the last 18 months, we have seen the same pattern repeat itself: 80-90% of raw keyword volume gets killed at the filter step, and the team wonders why their cluster outputs feel thin. Why naive filtering falls short
Tier 0a regex negatives and Tier 0b competitor-brand filters are free and deterministic. But Tier 1 (embedding similarity vs gold examples) and Tier 2 (LLM 3-axis classifier) are where the budget goes. Most teams over-rely on Tier 2 because they do not trust Tier 1’s threshold. So they ship every borderline keyword to the LLM, blow the cost ceiling, and the workflow silently exits at 50%. Our 3-layer defense
Layer 1: Postgres regex negatives at ingestion. Cheap, deterministic, ~3% precision gain.
Layer 2: Embedding similarity vs gold examples with adaptive threshold. We let clients tune the threshold per their niche, not a global default.
Layer 3: LLM classifier only on borderline cases. With cost gates that auto-pause at 80% of monthly budget. The result
Clients who implement this layered approach see 2-3x more clusters in the final brief, with 40% lower LLM spend. The filter step becomes a feature, not a bottleneck.
This article was generated as part of an end-to-end pipeline smoke test (2026-07-14) for Webley Media. SEO + GEO optimized for the visibility service pillar.