Low-latency New Data Notification

Low-latency notification when Calcbench processes new filings from the SEC and publishes data.

Calcbench pushes messages onto a queue when we publish new data. Your process listens on the queue. When you recieve a message you call the Calcbench API to retrieve data then add the data to your pipeline.

Implement the listener as below in a script that runs in an always on daemon managed by something like systemd.

If your system is down, messages will remain in the queue for 7 days. If you process throws an exception prior to completion the messages will be put back on the queue.

Warning

This requires Calcbench to create a subscription for you.

Requires the azure-servicebus package, install it with optional Listener dependency:

$  pip install calcbench-api-client[Listener]
calcbench.handle_filings(handler, connection_string='Endpoint=sb://calcbench.servicebus.windows.net/;SharedAccessKeyName=public;SharedAccessKey=Cb7VhLR6eJxsurCSPtXQHOJvlkU84CCCx2oB+T/so6Q=', subscription_name=None, topic='filings')

Listen for new filings from Calcbench

https://github.com/calcbench/notebooks/blob/master/filing_listener.ipynb.

Parameters:

  • handler (Callable[[Filing], None]) – function that “handles” the filing, for instance getting data from Calcbench and writing it to your database

  • connection_string (str) – azure service bus connection string

  • subscription_name (Optional[str]) – service bus subscription, Calcbench will give this to you

Usage::
>>> def filing_handler(filing):
>>>     if not filing.standardized_XBRL:
>>>         return
>>>     accession_id = filing.calcbench_id
>>>     data = point_in_time(
>>>             accession_id=accession_id
>>>             all_face=True,
>>>             all_footnotes=True,
>>>             )
>>>     print(data)
>>> subscription = "talk to calcbench to get a subscription"
>>> handle_filings(
>>>     filing_handler,
>>>     subscription_name=subscription,
>>> )
class calcbench.filing.Filing(**data)

A filing with the SEC or a wire press-release

everything except filing_id is basically optional becuase that is all you need to get the standardized data when it comes through the service bus.