How to Build a Real-Time Competitor News Feed with a REST API

Building an in-product competitor news feed requires solving three technical problems: finding updates for companies that do not have stock ticker symbols, removing duplicate press releases that reprint across dozens of outlets, and normalizing unstructured web articles into clean, typed JSON.

Traditional financial news APIs only index public tickers on major stock exchanges. Raw web scrapers and RSS readers, on the other hand, dump duplicate stories, broken formatting, and promotional clickbait directly into your database. If you want to show your users what their competitors, customers, or portfolio companies are doing, you need an entity-native news API that supports private companies, bundles duplicate reporting, and categorizes announcements automatically.

In this guide, you will learn how to build a production-ready competitor news feed using the Distill REST API. We will cover querying news by website domain, filtering by business impact, handling cursor-based pagination for infinite scroll, and rendering deduplicated story clusters with TypeScript and clean HTML.


Architecture overview: How the news feed pipeline works

When building a live news component into a B2B SaaS platform, CRM, or client dashboard, your client application should never query external news sources directly from the browser. Doing so exposes your API secret keys and prevents effective caching.

Instead, use a lightweight proxy architecture:

[Client Browser / Mobile App]
             │
             │  1. GET /api/competitors/stripe.com/feed?cursor=...
             ▼
   [Your Backend Service]  ◀─── [Redis Cache (15-minute TTL)]
             │
             │  2. GET https://api.distillintelligence.com/v1/companies/stripe.com/news
             │     Header: Authorization: Bearer <DISTILL_API_KEY>
             ▼
     [Distill REST API]
             │
             │  3. Returns deduplicated JSON (articles, updates, sources)
             ▼
   [Your Backend Service]
             │
             │  4. Formats payload & caches response
             ▼
[Client News Feed UI Component]

This architecture keeps your private API keys secure on your server, allows you to cache news items in Redis so multiple team members viewing the same competitor do not trigger redundant requests, and lets you append internal metadata (such as CRM deal stages or internal account tags) before returning the feed to the frontend.

Technical requirementTraditional news APIs (Finnhub, NewsAPI)Distill REST API
Company identifierPublic ticker symbol only (e.g. AAPL, MSFT) or broad keyword stringsStandard website domain (e.g. stripe.com) or numerical company ID
Private company supportMinimal or none. Unlisted startups return empty arrays or noisy string collisionsFull native coverage across startups, private enterprises, and public corporations
Source channelsSyndicated media wires (PR Newswire, Reuters) onlyMulti-channel: Press coverage, official company blogs, newsrooms, and LinkedIn posts
DeduplicationNone. Returns 30 individual items for a single syndicated announcementAutomated story bundling: 1 canonical event with clustered publisher sources
Noise reductionManual keyword exclusion queries requiredBuilt-in business impact scoring (low, medium, high) and category tags

Step 1: Authenticate and fetch news by company domain

The Distill API uses standard Bearer token authentication. Every request must include your API key in the Authorization header.

To retrieve news for a company, you do not need to look up an exchange code, CIK number, or stock ticker. You can query the company directly using its primary website domain via the /v1/companies/{companyIdentifier}/news endpoint.

Here is a basic cURL request fetching the latest updates for Stripe:

curl -X GET "https://api.distillintelligence.com/v1/companies/stripe.com/news?limit=10" \
  -H "Authorization: Bearer YOUR_DISTILL_API_KEY" \
  -H "Accept: application/json"

In TypeScript or Node.js, you can structure this call with a typed client function:

interface NewsFetchOptions {
  domain: string;
  limit?: number;
  minImpact?: 'low' | 'medium' | 'high';
  categories?: string[];
  cursor?: string;
}

export async function fetchCompanyNews(options: NewsFetchOptions) {
  const { domain, limit = 20, minImpact = 'medium', categories, cursor } = options;

  const url = new URL(`https://api.distillintelligence.com/v1/companies/${encodeURIComponent(domain)}/news`);
  url.searchParams.set('limit', limit.toString());
  url.searchParams.set('min_impact', minImpact);

  if (categories && categories.length > 0) {
    url.searchParams.set('categories', categories.join(','));
  }

  if (cursor) {
    url.searchParams.set('cursor', cursor);
  }

  const response = await fetch(url.toString(), {
    method: 'GET',
    headers: {
      'Authorization': `Bearer ${process.env.DISTILL_API_KEY}`,
      'Accept': 'application/json'
    }
  });

  if (!response.ok) {
    throw new Error(`Distill API error: ${response.status} ${response.statusText}`);
  }

  return response.json();
}

Why domain-based lookups matter: If you allow your users to monitor an unlisted competitor or an emerging software tool, they do not have a ticker symbol to input. Allowing users to enter a domain like linear.app or supabase.com immediately resolves to the correct corporate entity without ambiguous keyword searches.


Step 2: Filter out noise with impact ratings and categories

The primary failure point of user-facing news feeds is alert fatigue. If an in-app feed shows routine automated market recaps, SEO content marketing pieces, or minor directory listings, users quickly learn to ignore the widget.

The Distill API provides two built-in parameters to control content quality before records reach your application:

1. Business impact thresholds (min_impact)

Every news item is calculated with an impact level:

  • low: Routine operational updates, minor directory submissions, and general marketing pieces.
  • medium: Notable product launches, new executive appointments, strategic partnerships, and substantial press coverage.
  • high: Major funding rounds, acquisitions, critical leadership changes, major regulatory actions, and critical industry developments.

For an executive dashboard or customer-facing feed, set min_impact=medium or min_impact=high. This automatically drops low-signal noise without requiring you to write custom regex filters.

2. Event categories (categories)

Distill classifies every story into one or two operational categories, the best fit first, whether the company published it on its own blog, newsroom, or LinkedIn channel (returned with type="update") or the press covered it (type="article"):

  • Financials (Results, guidance, dividends, buybacks)
  • Funding (Funding rounds, listings, share and debt offerings)
  • M&A (Acquisitions, mergers, asset sales)
  • Contracts (Customer wins, supplier and government contracts)
  • Partnerships (Strategic alliances, co-selling agreements, integrations)
  • Product (Feature launches, version updates, pricing, deprecations)
  • Expansion (New office openings, international entries, new capacity)
  • Leadership (C-suite hires, board appointments, leadership departures)
  • Workforce (Hiring surges, layoffs, workplace changes)
  • Legal (Lawsuits, rulings, regulatory compliance)
  • Incidents (Data breaches, outages, recalls, insolvency)
  • Operations (Manufacturing changes, supply chain adjustments, restructuring)
  • Marketing (Campaigns, sponsorships, events, awards)
  • Sustainability (Climate targets, ESG, community programs)
  • Insights (Guides, research, commentary, interviews)

If your users only care about product launches and leadership changes, pass categories=Product,Leadership directly in your query parameters. The API returns stories in any of the categories you pass.


Step 3: Render clustered sources and deduplicated stories

When a company makes a major announcement, PR wires syndicate the release to dozens of publishers like Yahoo Finance, MarketWatch, and various regional news blogs. A naive news scraper returns 25 nearly identical records with slightly altered headlines.

Distill solves this at the ingestion layer through story clustering. The API returns a single canonical CompanyNewsItem, and places all secondary publisher coverage into a nested sources array.

Here is an example response payload showing a single canonical item with clustered source outlets:

{
  "data": [
    {
      "id": "comp_art_84712",
      "type": "article",
      "impact": "high",
      "title": "SpaceX successfully completes Starship orbital flight test",
      "snippet": "SpaceX completed a key flight test for its Starship launch system, validating thermal shield performance during atmospheric reentry.",
      "url": "https://www.reuters.com/aerospace-defense/spacex-starship-orbital-flight-test-2026-08-25/",
      "domain": "reuters.com",
      "name": "Reuters",
      "image_url": "https://www.reuters.com/images/spacex-starship-launch.jpg",
      "published_at": "2026-08-25T08:30:00Z",
      "categories": ["Product"],
      "original": null,
      "company": {
        "id": 136,
        "name": "SpaceX",
        "short_name": "SpaceX"
      },
      "sources": [
        {
          "id": "comp_art_src_991",
          "name": "Reuters",
          "domain": "reuters.com",
          "url": "https://www.reuters.com/aerospace-defense/spacex-starship-orbital-flight-test-2026-08-25/",
          "title": "SpaceX successfully completes Starship orbital flight test",
          "snippet": "SpaceX completed a key flight test for its Starship launch system...",
          "published_at": "2026-08-25T08:30:00Z"
        },
        {
          "id": "comp_art_src_992",
          "name": "TechCrunch",
          "domain": "techcrunch.com",
          "url": "https://techcrunch.com/space/spacex-starship-test-flight-milestone/",
          "title": "Starship reaches orbit in landmark test flight",
          "snippet": "The latest Starship test concluded with a splashdown in the Indian Ocean...",
          "published_at": "2026-08-25T08:45:00Z"
        },
        {
          "id": "comp_art_src_993",
          "name": "Bloomberg",
          "domain": "bloomberg.com",
          "url": "https://www.bloomberg.com/news/articles/spacex-completes-starship-test",
          "title": "Musk's SpaceX hits milestone with orbital test",
          "snippet": "SpaceX achieved a critical step forward for its deep-space transportation...",
          "published_at": "2026-08-25T09:10:00Z"
        }
      ]
    }
  ],
  "next_cursor": "eyJwdWJsaXNoZWRfYXQiOiIyMDI2LTA4LTI1VDA4OjMwOjAwWiIsImlkIjoxMjN9",
  "has_more": true
}

In your UI, you can display the primary headline and publisher, and add a clean secondary label showing coverage from additional outlets:

// Example component template logic
function NewsCard({ item }) {
  const secondaryCount = item.sources.length - 1;

  return (
    <div class="news-card">
      <div class="badge-row">
        <span class={`impact-badge ${item.impact}`}>{item.impact} impact</span>
        {item.type === 'update' && <span class="type-badge">Official update</span>}
      </div>

      <h4><a href={item.url} target="_blank" rel="noopener noreferrer">{item.title}</a></h4>
      <p>{item.snippet}</p>

      <div class="meta-footer">
        <span class="primary-source">{item.name} • {new Date(item.published_at).toLocaleDateString()}</span>
        {secondaryCount > 0 && (
          <span class="coverage-clustering">
            + reported by {secondaryCount} other {secondaryCount === 1 ? 'outlet' : 'outlets'}
          </span>
        )}
      </div>
    </div>
  );
}

Step 4: Implement cursor-based infinite scrolling

Traditional page-based pagination (e.g. page=1&page=2) is unreliable for real-time news feeds. If a new article is indexed while a user is reading page 1, requesting page 2 with offset pagination causes the items to shift, resulting in duplicate or skipped records on the screen.

The Distill API uses opaque, cursor-based pagination. When you query a news feed, the response includes two top-level fields:

  • has_more: A boolean indicating whether additional records exist beyond the current page.
  • next_cursor: An encoded string referencing the exact position and timestamp of the last record in the batch.

To fetch the next batch, pass the next_cursor value directly to the cursor query parameter on your next request. When has_more is false or next_cursor is null, you have reached the end of the available feed.

// Example infinite scroll state manager in TypeScript
export class CompetitorNewsStream {
  private domain: string;
  private nextCursor: string | null = null;
  public hasMore: boolean = true;
  public isLoading: boolean = false;

  constructor(domain: string) {
    this.domain = domain;
  }

  async loadNextPage() {
    if (!this.hasMore || this.isLoading) return [];

    this.isLoading = true;

    try {
      const url = new URL(`https://api.distillintelligence.com/v1/companies/${this.domain}/news`);
      url.searchParams.set('limit', '15');
      url.searchParams.set('min_impact', 'medium');

      if (this.nextCursor) {
        url.searchParams.set('cursor', this.nextCursor);
      }

      const response = await fetch(url.toString(), {
        headers: { 'Authorization': `Bearer ${process.env.DISTILL_API_KEY}` }
      });

      const payload = await response.json();

      this.nextCursor = payload.next_cursor;
      this.hasMore = payload.has_more;

      return payload.data;
    } finally {
      this.isLoading = false;
    }
  }
}

Step 5: Handle foreign-language news with automatic translation

If your users track international competitors (for instance, a European fintech company based in Stockholm or an automotive manufacturer in Tokyo), critical announcements often break in local languages first.

The Distill API automatically normalizes all incoming news into English by default:

  • The primary title and snippet fields are always provided in, or translated to, English.
  • If the source was published in another language, the payload contains an original object holding the native headline, native summary snippet, and the two-letter ISO 639-1 language code (e.g. de, ja, sv).

This structure allows you to build an intuitive UI where international news displays cleanly in English, with an option for bilingual users to toggle the original text:

function MultilingualNewsItem({ item }) {
  const isTranslated = item.original !== null;

  return (
    <article class="news-item">
      <h3>{item.title}</h3>
      <p>{item.snippet}</p>

      {isTranslated && (
        <details class="original-text-dropdown">
          <summary>Translated from {item.original.language.toUpperCase()}</summary>
          <div class="original-content">
            <p><strong>{item.original.title}</strong></p>
            <p>{item.original.snippet}</p>
          </div>
        </details>
      )}
    </article>
  );
}

Aggregating multiple competitors into a single team feed

Querying individual company domains works well when a user is viewing a dedicated company profile page. However, if you are building a central dashboard showing news across an entire competitive landscape (e.g. tracking 15 competitors at once), querying each company sequentially from your backend is inefficient.

The Distill API provides a dedicated Watchlist resource for multi-company aggregation:

  1. Create a watchlist in your Distill workspace containing all target competitor domains.
  2. Fetch the unified chronological stream using a single request to GET /v1/watchlists/{watchlistId}/news.

The watchlist news endpoint supports the exact same filtering parameters (min_impact, categories, since, until, and cursor pagination), but automatically combines and sorts updates across all monitored entities into a single feed.

// Fetch unified news across an entire watchlist of 20+ competitors
const response = await fetch(
  'https://api.distillintelligence.com/v1/watchlists/136/news?limit=25&min_impact=medium',
  {
    headers: { 'Authorization': `Bearer ${process.env.DISTILL_API_KEY}` }
  }
);

const { data: unifiedFeed, next_cursor } = await response.json();

Production best practices: Caching, webhooks, and rate limits

To deliver a fast user experience while keeping infrastructure costs minimal, consider the following best practices:

1. Implement a 15-to-30 minute Redis cache

Corporate news operates on an hourly cadence, not millisecond market ticks. Caching the JSON response of /v1/companies/{domain}/news in an in-memory key-value store like Redis for 15 to 30 minutes ensures sub-50ms render times for your users and prevents hitting rate limits when multiple users access the same company profile.

2. Use domain normalization before querying

Always normalize user input before sending it to the API. Strip protocols, trailing slashes, and leading subdomains so that https://www.stripe.com/pricing/ resolves cleanly to the root domain stripe.com.

3. Handle empty states gracefully

Early-stage startups or private companies in stealth mode may have weeks between public announcements or press mentions. If the API returns an empty data: [] array, provide a helpful UI message (e.g. "No notable news recorded in the last 30 days for this company") rather than showing a broken or blank card.


Frequently asked questions

Q1: Can I query unlisted startups and private companies with the Distill API?

Yes. Unlike financial market APIs that require a stock ticker symbol on an exchange, the Distill API allows you to query any public or private business using its primary website domain (such as stripe.com or openai.com).

Q2: How does the API prevent duplicate press releases?

Distill uses automated story clustering to bundle multiple media outlets covering the same event into a single canonical record. The API returns one primary headline and summary, with all syndicated publishers grouped inside a nested sources array.

Q3: What types of sources are included in the feed?

The feed ingests both third-party media coverage (trade publications, global business outlets) and direct company publishing channels, including official corporate newsrooms, company blogs, Substack publications, and official corporate LinkedIn posts.

Q4: How does pagination work?

The API uses cursor-based pagination with opaque tokens. When fetching a page, pass the next_cursor string returned from the previous request into the cursor query parameter to retrieve the next chronological batch without item shifting.

Q5: Is there a free trial to test the API?

Yes. You can test the API in your own stack before committing to anything. Request an API key to try it out, tell us a bit about what you're building, and we'll get you set up.


Start building your competitor feed in minutes

FeatureIn-house scraper pipelineDistill REST API
Setup time2 to 4 months of engineering timeUnder 15 minutes with standard REST endpoints
Maintenance burdenContinuous repair of broken website selectors and anti-bot blocksZero maintenance. Structured, normalized JSON payloads
Signal qualityRaw, duplicate-heavy RSS and HTML diffsImpact-scored, deduplicated, and auto-translated
Ongoing costHigh proxy costs, compute overhead, and developer hoursTransparent SaaS plans starting at $59/month

Integrating live market intelligence into your product does not require maintaining brittle web scrapers or subscribing to expensive financial terminals. With clean domain-based lookups, automated deduplication, and structured category filters, you can embed a high-signal competitor feed into your application in an afternoon.

To test the API with your tech stack, review the API documentation or request an API key to try it out.

Company news for any company, via API

Query news, press releases, blog posts, and LinkedIn updates for any company by its domain, deduplicated and scored by business impact.

Request API access