Skip to main content
Testing. You are viewing the public testing version of GCN. For the production version, go to https://gcn.nasa.gov.
New Announcement Feature, Code of Conduct, Circular Revisions. See news and announcements

Frequently Asked Questions

Kafka

What is Kafka?

According to the Apache Kafka web site, “Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.”

In recent years, Apache Kafka has seen wide adoption by the astronomy community, notably by the Vera C. Rubin Observatory. GCN operates a highly available cluster of Kafka brokers in the cloud on Amazon Web Services that distributes GCN Notices to users.

The Kafka platform itself does not care about the format of the records. However, Avro and JSON are very common formats for Kafka records because of a rich ecosystem of open-source and commercial tools for them built on Kafka (for example: Kafka Streams, ksqldb, Kafka Connect). The Vera C. Rubin Observatory's alert system uses Avro over the Confluent Wire Format. GCN has developed a unified schema for communicating Notices as JSON records.

How do I start receiving GCN notices via Kafka?

See Kafka Client Setup.

What ports do I need to open in order to receive or send GCN notices with Kafka?

Clients connecting to GCN only need to be able to make outbound (egress) TCP connections. The client connects to the following hosts and ports.

DirectionProtocolPurposePortHost
outboundTCPHTTPS443auth.gcn.nasa.gov
Kafka9092kafka.gcn.nasa.gov
kafka1.gcn.nasa.gov
kafka2.gcn.nasa.gov
kafka3.gcn.nasa.gov

What does the warning Subscribed topic not available: gcn.classic.text.AGILE_GRB_GROUND: Broker: Unknown topic or partition' mean?

This warning means that there have not been any recent alerts on that topic.

As a GCN Notice producer, what is the largest message that I can send over Kafka?

The GCN Kafka brokers will accept messages up to about 4 MiB (4,194,304 bytes) in size. Please contact us if your mission requires larger messages.

Note that if your messages are 1 MB (1,000,000 bytes) or more in size, then you will need to set the message.max.bytes client configuration property when you create your Kafka producer. For example, in Python you would set that property as follows:

from gcn_kafka import Producer
producer = Producer(
    client_id='fill me in',
    client_secret='fill me in',
    **{'message.max.bytes': 4194304})

See GitHub issue confluentinc/librdkafka#3125.

How do I receive GCN Notices via email from GCN Classic over Kafka?

To get started, sign in or sign up and then select 'Email Notifications' from account dropdown menu. See also GCN Circular 32517.

Circulars

Why do GCN Circulars that I submit by email appear to be double spaced?

Some email clients in some configurations are known to add extra line breaks to emails. When you are preparing to submit a GCN Circular by email, you should always make sure that you are composing a plain text message. See the following instructions for various mail clients:

Note that it used to be common practice for Circulars submitters to add line breaks to manually wrap long paragraphs in GCN Circulars. This practice is no longer recommended in the GCN Circulars style guide.

Accounts

How do I sign in as a legacy GCN Classic Circulars user?

If you had a GCN Circulars account prior to April 17, 2023, your account was migrated to the new GCN. Your receiving preferences, submitting permission, name and affiliation will be associated with your account as soon as you sign in for the first time. If you choose to sign in via email address and password, tap "Sign up" not "Forgot your password?".

Operations

Does GCN keep working during a U.S. federal government shutdown?

Yes. During a U.S. federal government shutdown, all GCN services (GCN Circulars, GCN Notices, Kafka cluster, the web site) remain fully operational. However, there may be the following minor impacts to GCN users:

Looking for U.S. government information and services? Visit USA.gov