This article is based on material authored by members of the
news.newusers.questions
Moderation Board and nnq-workers mailing list,
particularly by Jon Bell (until 2005)
and Thor Kottelin (since 2007).
What newsgroups are and how they work
Concisely, newsgroups are a means of public discussion.
Newsgroup articles (messages) look like email,
but millions of people all over the world can read them.
This note describes how newsgroups work in the sense of what happens
to articles after they are posted. In order to find out how newsgroups work in
the sense of how to use your news-reading software, please look for documentation
for that software or post a question to an appropriate newsgroup.
Questions and answers
Can someone give a brief, not-too-technical description of where
a message goes when I post it into a newsgroup?
Newsgroup articles are distributed via news servers, which contain databases of articles.
Internet service providers (ISPs), schools and other organisations operate news servers.
Do all the messages travel to a central site,
and do all the sites that want it pick it up there?
No, there is no central server on Usenet. A newsgroup article
propagates from one server to another, starting from the server where
it is first posted.
Moderated newsgroups are sort of an exception, in that all articles are
first forwarded via email to a moderator for approval. The moderator
posts them on his or her news server; from there, they propagate as
described below. Different moderators use different news servers.
Do all messages travel to all news servers?
Ideally, all articles in a newsgroup would travel to all news servers
that carry the newsgroup. When you post an article, the
result is tens of thousands of copies, all over the world.
More specifically, when you post an article, it goes first to your local news server
(operated e.g. by your ISP or school).
Your server then sends copies of the article to its "neighbours",
that is, to servers with which it has agreed to exchange articles.
Those servers, in turn, send copies to their neighbours.
Eventually, every server that carries the newsgroup has a copy.
In what order?
Most servers normally forward articles more or less in the order of arrival.
This sequence can become scrambled for various reasons,
which is why you often see responses before the original article arrives.
How do they know which servers want them?
News server administrators arrange among themselves which newsgroups they exchange.
The receiving server's admin tells the
sending server's admin which newsgroups he or she wants to receive. The
sending server's admin then configures his or her server to send only those
newsgroups.
How do they know which sites they have already visited? How does a message
avoid the same site twice?
There are two methods. Servers usually use both of them, in sequence:
- The Path: header line shows the sites that the article has
travelled through, so far, between the originating server and the current server.
If the receiving server appears in the Path: line, the sending
server does not try to send the article, because it knows that the
receiving server already has received a copy.
- The Message-ID: header line contains an identifying code that is
different for every article. Before transmitting the article, the
sending server asks the receiving server, in effect, "Do you have an
article with Message-ID such-and-such?" The receiving server responds
either "No, please send me a copy" or "Yes, I have it already",
whereupon the sending server either sends the article or goes on to the
next one.
How long do messages stay posted?
Each news server removes old articles, usually once a day, to
make space for new ones. This is called expiring.
Most servers do this based on the number of days
an article has been on that server. The expiration time varies from one
server to another, and can vary from one newsgroup to another on the
same server. It might be less than a day
(such as for groups that carry binary content),
or it might be two weeks or more.
Therefore, even after an article expires from your own server, it is probably
still visible on many other servers.
Note, however, that even though you cannot see an article any more, it may
still be present on your server. Most news-reading software keeps track
of which articles you have read, and shows those articles to you only once. This way,
you do not have to wade through the same articles repeatedly.
There should be a command, button or something that "shows all",
"shows previously-read" or "unmarks" articles so that you can see all
the articles that have not yet expired from your server.
Behind the scene, how does a newsreader communicate with a news server?
An example of how an article posting takes place on the news transfer protocol level
is available on the
"How to test an
NNTP (news) server using telnet" page.
You can score guru points with your friends by reading and posting news without using a newsreader. ☺
