NewsPlex for Windows
Version 2.8 July 2000
Copyright © 1999-2000 by Adam Mirowski

Introduction

NewsPlex is a news-server multiplexer. It allows any news-reader to access several news-servers simultaneously, by merging their content transparently into one single news-server. NewsPlex merges the lists of news-groups proposed by different news-servers into one single list, and for each news-group, it merges the lists of articles available on different news-servers into one single list of articles. NewsPlex does not retrieve articles before the news-reader requests them. It only creates local indexes and redirects article retrieval as necessary.

Additionally, NewsPlex can be used to quickly explore large lists of candidate news-servers and evaluate their service value. This is called the Minimal mode.

NewsPlex runs on your own machine (or possibly on some other machine) and appears as a standard news-server to your news-reader and as a news-reader to your news-servers. It is therefore somewhat similar to a proxy or to a web cache.

NewsPlex can be controlled thru command-line options at startup time, or dynamically, from a Web browser or by telnetting into it.

NewsPlex runs on Windows (95/98/NT/2000), OS/2 and Unix (Linux and Solaris). This version and this manual are for Windows.

Other features

NewsPlex can service several news-readers at the same time, which notably allows to control it from the network while it is already being used by some news-reader.

NewsPlex can be seamlessly inserted between your existing news-server and your news-reader, without disturbing the list of news-groups or the article numbers.

NewsPlex always uses the best news-server for requesting an article, among those having it. It will use the remaining news-servers if the best one could not deliver the article. Servers are rated dynamically according to speed, latency and availability.

NewsPlex will close the connection to the news-reader in case of errors appearing in the middle of an article retrieval, typically news-server timeouts. The NNTP protocol limitations do not allow the recovery of such situations. NewsPlex will however remember the error condition, and if the article is asked for again, it will try to use a different news-server for obtaining it.

NewsPlex can be run in the background, permanently. It will explore news-servers periodically to maintain its article database up to date.

NewsPlex does not need to be stopped if the network connection goes down, for example because you disconnect from your ISP. It will wait until the connection is up again and continue operations.

NewsPlex is able to use any news-server as news source, even news-servers which implement very old versions of the news transfer protocol, and yet always exports a modern interface to news-readers. Errors occurring during the exploration of news-servers are also recovered, thru either restarts, deferring or workarounds.

NewsPlex supports connecting to news-servers either directly, or through proxy servers (SOCKS 5, SOCKS 4 or Sun-specific).

NewsPlex has simple spam filtering possibilities based on the From and Message-ID fields of article descriptions. Wildcards are supported.

NewsPlex supports user and priviledged user login/password authentication.

Asynchronous article retrieval

NewsPlex supports asynchronous article retrieval. Instead of providing an article immediately, NewsPlex can instead simply remember that it has been requested and retrieve it independently and in parallel with the news-reader operation. The news-reader is returned a fake article, which only informs about the asynchronous download in progress. The real article will be available in the async news-group, managed locally by NewsPlex, as soon as the download finishes. Having asynchronously retrieved articles visible in a separate news-group allows to use the news-reader facilities for managing multi-part postings. NewsPlex slightly changes the Message-IDs inside asynchronously retrieved messages, so that news-readers able to detect cross-posts consider them as different from the fake articles returned initially.

Asynchronous article retrieval allows to accelerate downloads by parallelizing operations and eliminating idle moments.

An arbitrary number of article retrievals can be scheduled, and up to 4 downloads are performed at once (this can be adjusted with the -a command-line option). NewsPlex remembers which articles remain to be retrieved even across shutdowns and restarts and finishes downloads the next time it is run.

Note that zombie articles (cf. below) are considered as permanently unavailable.

When to use NewsPlex?

NewsPlex is useful when your main news-server does not offer all the news-groups you want to read, or when these news-groups are incomplete (for example some articles are filtered out or parts of multi-part posts are missing), or when the expiration delays for articles are too short.

If your news-servers are unreliable, become easily unreachable or give timeouts, NewsPlex will isolate you from these errors, perform the necessary retries in the background and keep you posted with the latest news in the minimal time and without drudgery.

Few news-readers allow using several news-servers at the same time, and no news-reader known to me can take articles from several news-servers for a single news-group transparently, or request an article from different news-servers until successful. NewsPlex allows you to choose your news-reader on other criteria than these abilities.

Author

Adam Mirowski <mir@sun.com>

NewsPlex version 2.8 is freeware. It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. You can use at your own risk, and report whatever problems you have encountered with it. You are free to copy this software ONLY if you include this document file and all the other accompanying files with it. You may NOT charge anyone for a copy of this software other than a small copying fee. You may NOT include this software with any commercial software without the written consent of the author.

Installation and configuration

Unpack the NewsPlex distribution archive to any directory. You should have one executable file newsplex.exe and one help file newsplex.htm in HTML format, which you are currently reading. There should also be a boilerplate file file_id.diz and a descript.ion file with descriptions of all the files in the distribution.

NewsPlex requires one subdirectory and two files in it to start. They must be located either in the directory from which NewsPlex is started or in the directory indicated thru the -d command-line option. This directory becomes the work directory.

Filename
Description
config/servers This text file contains the list of news-servers to use and identifies the main one. It can also specify which proxy, if any, should be used for accessing a given news-server (see a specific chapter below).

Each line describes a single news-server and has following format:

server-number server-url [flags] [# comment]

Example:

1  news1.yourisp.com			# server without login
-2 mylogin:mypass@news2.yourisp.com	# server with login (the main server)
3  news3.yourisp.com:4000		# server on port 4000
# this is a comment
	

server-number must be unique for each news-server and greater than 0, except for the main news-server entry. It identifies the news-server in the article database and cannot be changed after a news-server has been explored.

If the server-number is negative, the news-server will be considered as the main news-server. Only one news-server can be main.

If you need to pass funny characters in the login or in the password, use standard % escape sequences: a % character followed by the hexadecimal ASCII number of the offending character. For example, @ can be escaped as %40, : as %3A and as blank as %20. So if the login is for example ano:nymous and the password joe@domain.com, the resulting URL will be
ano%3Anymous:joe%40domain.com@newshost.com.

flags are undocumented for now; do not set them, with one exception. If in the news-server exploration logs you see that for certain news-servers NewsPlex keeps exploring certain articles ranges without getting any article descriptions, set flags to 4 for the news-server. This however introduces a penalty: if the news-server generates article descriptions out of order and at the same time declares the numbers in advance, you might not get all the article descriptions.

It is possible to add comment lines, starting with a #, or to put comments at the end of lines, starting them with #.

A line containing just the word exit will make NewsPlex ignore the rest of the file. A line containing just the word dontupdate makes NewsPlex not automatically update the news-servers which follow.

config/groups.yes This file contains the names of news-groups which should be retrieved from news-servers and offered to news-readers. The names of news-groups can contain * and ? wild-cards. Eg:

		*midi*
		*music*
	

Think twice before putting a sole * on a line into the config/groups.yes file, since this will make NewsPlex take all news-groups from all the news-servers, and create a really huge article database. About 200 kilobytes are required in average to store article descriptions for a single news-group.

You should select your existing news-server as the main news-server. NewsPlex will then synchronize with it when run for the first time, so as to provide the same news-groups and article numbers. This way, your news-reader will not be confused about which articles have already been read and which have not. NewsPlex will then complete the list of news-groups and articles with what other news-servers offer.

The main news-server is used for posting articles by default. Additionally, the NPPOSTSERVER command can be used to change the posting news-server globally at any time, or to view the current selection.

If you do not select any news-server as main, NewsPlex will consider all the news-servers as equivalent, explore them simultaneously, and number articles starting from 1 up.

Three further files are optional:

Filename
Description
config/groups.no
(optional)
This file indicates the names of unwanted news-groups which might get selected by config/groups.yes. No wild-cards are allowed here. Eg:

		alt.music.boring
		alt.midi.lame
	
config/groups.fix
(optional)
This file indicates the news-groups which should never be filtered out by the config/groups.no file. This is mostly a security against putting wrong content into the config/groups.no file. Eg:

		alt.music.best
		alt.midi.best
	

If a news-group vanishes from a news-server, this file allows to classify it as Wanted if it was listed there, Unwanted if it is now in config/groups.no, or Accepted if it was in neither of these files.

config/groups.fix is also used to generate information about the quality of a news-server: what percentage of interesting news-groups it provides and what percentage of its news-groups are interesting.

config/authinfo
(optional)
This file is used if the -A option is passed on the command line. It can define the login names and passwords which should be used for protecting access to NewsPlex. It should contain one or two lines of the following form:
	user        username  password
	supervisor  username  password
The user line is equivalent to passing the -U option on the command line, and the -P option is equivalent to passing the -P option. The last occurrence of a given line type is retained.

If you modify any of these files, you will need to restart NewsPlex before the modifications are taken into account. The restart can be performed with the NPRESTART command or by shutting down NewsPlex and re-running it.

Operation

NewsPlex is a console-mode program. It can be conveniently started from either a command-line window or from a shortcut on the desktop. NewsPlex does not need command-line parameters to operate. After you have setup the config files you can start NewsPlex by typing its name. Then you can connect your news-reader to machine name localhost (or Internet address 127.0.0.1) and the standard NNTP port 119. Note that NewsPlex will first need to explore at least one news-server before offering news-groups and articles.

To shutdown NewsPlex when it runs in the foreground you can send it a Ctrl-C or a Ctrl-Break signal. You can also click the Close button, double-click the system menu button or select Close from the system menu of the NewsPlex window.

It is also possible to stop NewsPlex from the Web browser interface or by telnetting into it (telnet localhost 119) and entering the NPSHUT command.

As the last recourse, you can terminate NewsPlex using the End Program button of the Task Manager (Ctrl-Alt-Del dialog).

Proxies

By default, NewsPlex attempts to connect to news-servers directly. It is however possible in the config/servers file to specify a proxy for each news-server. Several proxies can be used at the same time, and each news-server can be accessed thru a different proxy if necessary.

First, define a list of proxies you intend to use, with entries like this:

proxy proxy-url,proxy-group-number

The proxy-url must start with the proxy type, either none, or socks5://, or socks5b://, or socks4://, or sun://.

none means that no proxy should be used and the connection can be established directly. This is the default way of connecting news-servers.

If the type is not none, the proxy-url should then specify the name of the proxy server and potentially its port as well as the login and the password if necessary. The default port is 1080 for socks5, socks5b and socks4 servers and is 3666 for Sun servers. By default no login or password information is sent.

The proxy-group-number must be a small integer number. It allows to identify a proxy and to reference it from a news-server entry. The default proxy-group-number is 0.

Note: socks5b servers are buggy socks5 servers. If you observe that NewsPlex is unable to connect at NNTP level to any news-server, and the logs/results file shows truncated welcome messages from them, with missing first 6 chars (eg. instead of 200 newsservername... you see wsservername..., and instead of 502 You are not allowed to talk you see u are not allowed to talk, use socks5b instead of socks5 to define the type of your proxy.

For each news-server entry, it is possible to reference the proxy in the server-url by appending a final ,proxy-group-number to it. By default, value 0 is assumed. To quickly modify a config/servers file used on a direct Internet connection so that it can be used on a proxy-ized Intranet connection, just add an initial line defining proxy 0. Example:

proxy socks5://login:pass@socksproxy:1080,0
proxy none,1
proxy sun://oursunproxy,2

1  news1.yourisp.com,0			# server outside, will go thru proxy 0
2  mylogin:mypass@companyserver,1	# server on local network, no proxy
3  news3.yourisp.com:4000,0		# another outside server, use proxy 0
4  news4.someisp.com,2			# go thru sun proxy 2 to reach this
Note that all ,0 are actually unnecessary.

If no proxy is defined in proxy group 0, NewsPlex will automatically assume a direct connection for this proxy group, that is, create an implicit proxy none,0 entry. This allow to not define any proxies in the config/servers file if the connection is always direct.

Proxy groups

Several proxies can have the same proxy-group-number and hence constitute a proxy group.

NewsPlex will use the first proxy defined in a given proxy group, except if the news-server indicates that it does not allow additional connections from this proxy (or from your host, if the proxy is none).

In such a case, NewsPlex will try to use the next proxy from the same proxy group, so that the news-server thinks a different news-reader is connecting. NewsPlex will also try to use a different proxy from the same proxy group if the current proxy indicates that it does not authorize establishing the NNTP connection. NewsPlex will not try to use another proxy if the news-server is simply not reachable from the first proxy in a proxy group.

Proxy chains

Sometimes, it is not possible to reach news-servers using a single proxy. For example, this proxy might not allow establishing NNTP connections but might be the only way out of an intranet.

To deal with this situation, NewsPlex supports proxy chains. This means that it can use a proxy to reach another proxy, and then use this second proxy to actually connect to a news-server.

After proxies have been defined in the config/servers file, you can chain them using the proxychain directive:

proxy socks5://intranetproxy,0
proxy socks5://externalproxy,1
proxychain 0 1

1  news.somedomain.com,0
Here, proxy group 1 is chained behind proxy group 0. Every connection going thru proxy group 0 will transit thru proxy group 1 before reaching the news-server. Referencing proxy group 1 directly remains possible.

Only two components are allowed in proxy chains. A single proxy group can appear only once in the first position of a proxy chain.

If the second proxy group has several proxies, only the first one defined will be used. However, all proxies in the first group will be tried if necessary.

Proxy chains are longer to establish and slower in operation than ordinary proxy-based connections, which themselves are worse than direct connections. They also require that the first proxy does not forbid connecting to the second one. In the above example, intranetproxy must allow connecting to port 1080 on externalproxy.

Produced files and directories

When operating, NewsPlex produces a series of files and directories in the work directory.

Filename
Description
database The database directory contains the article database. It contains one file for each news-group offered by NewsPlex. Each file has one line per article. It describes the article and indicates which news-servers have it. Do not modify the files there.
async The async directory is used when asynchronous article retrieval is enabled. It stores both control files and the articles themselves.
async/scheduld The async/scheduld file contains the list of articles which are being retrieved asynchronously. It is deleted, together with the async/done file, after all articles have been processed.
async/done The async/done file contains the list of asynchronously downloaded articles which have been completely processed. It is deleted, together with the async/scheduld file, after all articles have been processed.
async/summary The async/summary file stores the log of asynchronous article retrievals. Both successful and unsuccessful attempts are logged there after they have completed. For a given article, there can be several failed logs before the successful one. Or the last log can be a permanent failure, eg. article deletion from all news-servers which used to have it.
async/xxxxxxxx.ok The async/xxxxxxxx.ok files, where x are hexadecimal digits, are asynchronously retrieved articles. They are also visible in the async news-group generated by NewsPlex locally. They are deleted after being successfully retrieved for the first time by a news-reader.
If an article could not be retrieved, for example because it has been deleted from all news-servers, then the async/xxxxxxxx.ok file contains only an error message from NewsPlex, formatted as a news article.

Note that any async/xxxxxxxx.ok file, where x are hexadecimal digits, copied into the async directory, will be visible in the async newsgroup and can be retrieved from a news-reader. After the first successful retrieval, it will be deleted.

async/xxxxxxxx.log The async/xxxxxxxx.log files, where x are hexadecimal digits, are logs of asynchronous article retrievals. They exist only while the article retrieval is in progress and when it completes, they are appended to the async/summary file.
async/xxxxxxxx.tmp The async/xxxxxxxx.tmp files, where x are hexadecimal digits, are articles which are being received asynchronously. After the article retrieval is finished, articles are renamed to async/xxxxxxxx.ok and visible in the async news-group generated by NewsPlex.
async/xxxxxxxx.bad The async/xxxxxxxx.bad files, where x are hexadecimal digits, are articles which could not be retrieved completely because of temporary errors. These files always have a size greater than zero, otherwise they are not kept. During the next article retrieval attempt, they will be deleted and replaced with async/xxxxxxxx.ok files.
etc The etc directory hosts the various output files that NewsPlex produces and sometimes reuses.
etc/groups.new The etc/groups.new file logs the new news-groups appearing in NewsPlex. It is presently written but never read, and can be deleted at any time. Each line of the etc/groups.new file contains the date of news-group addition, expressed in seconds since 1/1/1970, and the name of the news-group.
etc/popular The etc/popular file records news-group popularity, which is measured as the total number and size of articles retrieved from the news-group. When exploring a news-server, NewsPlex first asks for the most popular news-groups. Each line of the etc/popular file contains the number of articles requested, the total size in kilobytes of these articles, the date of last article retrieval expressed in seconds since 1/1/1970 and the name of the news-group.
etc/retrievd If the -R command-line option is passed, NewsPlex stores in the etc/retrievd file the Message-ID fields of all the articles which have been successfully retrieved by news-readers. This file is also used when the -M command-line option is specified (see section below).
explore The explore directory stores the log files resulting from the exploration of news-servers. Log files are named after news-servers.

Each news-server exploration has the same scenario, which is logged into the file:

  1. NewsPlex asks for news-groups matching patterns in config/groups.yes
  2. NewsPlex produces a resulting list of interesting news-groups. The first column is the number of articles, the 2nd column is news-group name.
  3. NewsPlex displays which news-groups have vanished compared to the previous run.
  4. NewsPlex explores each news-group, using the newsrc file to avoid retrieving the same article descriptions twice and using the statistics from the etc/popular file to explore the most popular news-groups first. Groups which have never been visited are explored in a round-robin order, thanks to the timestamp info from the newsrc file.
This directory is preserved when NewsPlex terminates. The files inside can be deleted at any time, except when locked by NewsPlex because used. If a log file grows too big, a new file is started and the old one is renamed into .bak. The previous .bak file is deleted.
journal The journal file logs certain events and conditions as NewsPlex runs, like starts of news-server explorations and incoming connections from news-readers. This file is renamed to .bak each time NewsPlex is started and this file is over a certain size limit and the previous .bak, if any, is deleted.

When NewsPlex terminates, it prints into the journal file some summary information about the usage of news-servers during the session, similar to the results of the NPSERVERS command.

list The list directory temporarily stores the full news-group listings from news-servers which cannot return news-groups matching patterns. It is automatically destroyed when NewsPlex exits.
logs The logs directory hosts the various output files that NewsPlex produces but does not reuse. Its content can be deleted at any time.
logs/cliXXX The logs/cliXXX files log commands issued by your news-reader and the responses it got. XXX is a unique number. Once a news-reader has disconnected, the contents of its logs/cliXXX file are appended to logs/clients and the logs/cliXXX file is deleted.
logs/clients The logs/clients file logs closed sessions from news-readers. It can be deleted at any time.
logs/results The logs/results file indicates the exploration results for news-servers. Each line has this format:

		server-url [flags] # result (comment)
	

This file is kept when NewsPlex terminates. It can be deleted at any time.
Note that this file has the same format as a raw config/servers file (see the -r command-line option below). Notably, if you have defined proxies for accessing the news-servers, they are copied into this file.

logs/unusable The logs/unusable file is only created in Minimal mode (-m command-line option). It contains the logs of news-server explorations for all news-servers which did not have any interesting news-groups or which could not be contacted at all. It can be deleted at any time.
logs/intruder The logs/intruder file logs rejected incoming connections. For each connection, the date, the IP address, the IP port and the host-name are stored. The file can be deleted at any time, but you better check from time to time who tries to use NewsPlex on your machine without authorization.
newsrc The newsrc directory contains one file per news-server used. It logs which news-groups have been explored on the news-server and which article descriptions have been retrieved from them.

The format of the files in the newsrc directory and the information in them are not quite the same as for the newsrc or .newsrc files used by news-readers. Lines starting with a tab contain the name of a news-group and the date of its last exploration, while the other lines contain the ranges of article numbers for which article descriptions have already been retrieved.

You can delete a file corresponding to a news-server to force NewsPlex to retrieve the full article list for each news-group at the next exploration of the news-server. This will however almost certainly create dangling references in the article description database. Such dangling references can be removed with the NPCHECKGROUPS command, preferably after the news-server has been re-explored.

panic The panic file is created if NewsPlex detects an non-recoverable internal error. Once a short message has been logged into this file, NewsPlex exits.

Commands

NewsPlex supports most of the standard NNTP commands for servicing news-readers: ARTICLE, AUTHINFO USER, AUTHINFO PASS, BODY, GROUP, HEAD, HELP, LAST, LIST, LIST ACTIVE, LISTGROUP, MODE READER, NEWGROUPS, NEXT, POST, QUIT, SLAVE, STAT, XHDR and XOVER.

Note that although the STAT command could be executed entirely locally, it actually goes to news-servers in order to allow validating the availability of an article.

NewsPlex also adds a few commands of its own, which are not intended for usage by news-readers but rather by an operator. These commands can be entered after performing telnet localhost 119 on your computer. Note that the HELP command does not display them if the user is unprivileged.

Conversely, it is possible to enable the Web browser interface in NewsPlex by passing the -p port option on the command line, and connect from any Web browser to url http://localhost or to url http://localhost:port if port was not 80.

Command
Description
NPACCESSED Display which news-groups are currently being accessed by news-readers or are being updated.
NPASYNC [min-lines] Enable asynchronous article retrieval for articles longer than min-lines lines and which seem to contain binaries. If min-lines is preceded with a minus, all articles longer than min-lines are retrieved asynchronously. Passing min-lines of 0 disables asynchronous article retrieval, but does not stop article retrievals already scheduled. The NPASYNC command also displays the list of articles which are being retrieved asynchronously, followed by line giving their count and total number of bytes.
NPCHECKGROUPS [mode] Check the article database for coherency with the newsrc files and optionally remove dangling article references. Such references can appear if NewsPlex is brutally killed before having saved modifications done to a database file, while corresponding modifications done to the newsrc file have already been saved (note that the opposite situation simply makes NewsPlex retrieve some article descriptions again, as regets). mode specifies the operation mode:

0 - check only, silent mode
1 - check only, verbose mode
2 - fix errors, silent mode
3 - fix errors, verbose mode

If a news-server is being explored when this command is entered, no checking will be performed for it.

NPCLIENTS Display the list of currently connected client news-readers. For each client, the following information is displayed:
  • Socket number, which gives the log file name. The log file for socket XXX is named logs/cliXXX.
  • The IP address of the client
  • The IP port of the client
  • The hostname of the client
  • The time the client came in
  • The current group for the client. If it is between parentheses, the group is not loaded in memory yet.
  • The current article number in the group, or -1 if there is none.
  • The calling connection is marked with an asterisk in the last column.

This command does not presently have an equivalent in the Web interface.

NPCLOSE server-number Close all idle connections to a given news-server.
NPCONNECTED Display which news-servers are currently being connected to. This can be either for exploring or for retrieving an article on news-reader request. Connections are maintained a few minutes even when idle, so as to minimize the time necessary to re-use a news-server.
NPDELETEOLD days Delete article descriptions older than days days in all news-groups. Articles which do not have a recognizable Date will be kept.
NPDELETESERVER server-number Delete a news-server which is no more usable.

If a news-server becomes unusable, as the explore/server-name file indicates, you can delete it from NewsPlex using NPDELETESERVER server-number. NPGLOBALSTATSwill then not show any articles for it anymore. Then you can delete it from the config/servers file.

NPEXPLORESERVER server-number Explore a news-server immediately rather than waiting till its time comes. This command is also useful if NewsPlex has been started with automatic update disabled (-u).
NPFILTER Apply spam filters from config/filters to current news-group. After selecting a news-group with the GROUP name command, you can enter NPFILTER to apply the current set of filters from the config/filters file to this group. Note that other files from the config directory are only read on startup.
NPGLOBALSTATS Display provider stats for all news-groups.

NewsPlex will display for each news-server how many articles it has and how many of them are unique. If a given news-server does not appear in the list, NewsPlex has no articles from it.

NPGROUPSTATS Display provider stats for the current news-group. After selecting a news-group with the GROUP name command, you can enter NPGROUPSTATS to see the news-servers which contain articles for this news-group.
NPMARKRETRIEVED article-number Mark article article-number in the current news-group as retrieved (see section below).
NPPOSTSERVER [server-number] Define a news-server as the posting news-server. This command can be used to dynamically change the news-server set by the config/servers file. This is a global setting, valid for the current and all other connections. Without argument, this command displays the current posting news-servers.
NPRENUMBER level This command allows to help dealing with unavailable news-servers (asynchronous article retrieval is another way).

If no news-server having an article is reachable, by default NewsPlex will send to the news-reader

	500 No server having article is reachable 
Some news-readers however do not like this message and do not request further articles.

If NPRENUMBER is activated, NewsPlex will instead send

	423 no such article number in this news-group 
which makes the news-reader think the article has simply been deleted, so further articles will be retrieved correctly.

NewsPlex will however create a new reference for the article which will be seen by the news-readers the next time they asks for new articles. There are 2 possible ways for creating this new reference:

  1. Moving the article under a new number (NPRENUMBER 1).
    This is clean, but some news-readers check for cross-posted articles using Message-IDs and will ignore articles which have the same Message-ID as an old article, even if this old article has been signaled as deleted previously.

  2. Also changing the Message-ID (NPRENUMBER 2).
    This allows to deal with the previous issue. NewsPlex also alters the Message-ID to make the article look new. However, the article will not appear in the correct place in the thread to which it belongs: it will lose its followups.
It is possible to revert to the normal operation by sending NPRENUMBER 0.
NPRESTART Shutdown NewsPlex and restart it immediately. All configuration files will be re-read. The original command-line options will remain active.
NPSERVERS Display information about the usage of news-servers. For each news-server, NewsPlex will display whether it has already taken article descriptions from it during the current session or was able to connect at all. Other information is the time since the last exploration, the current speed, the number of articles and bytes retrieved and whether the news-server allows posting articles.
NPSHUT Shutdown NewsPlex. NewsPlex will terminate.
NPZOMBIEDAYS [zombiedays] Equivalent to the -z command-line option. Without argument, this command displays the current value only.

Command-line options

NewsPlex accepts the following command-line options at startup:

Option
Description
-a count Perform count asynchronous article retrievals at once instead of default 4, if asynchronous article retrieval is enabled. This parameter should be equal to the speed of your Internet connection divided by the average speed of your news-servers.
-b Run in background.

By default, NewsPlex runs in foreground and blocks the window from which it has been started. The -b option makes NewsPlex put itself into the background. On Windows 95, this frees the window and allows to run other programs from it.

On Windows NT, NewsPlex must be started with start NewsPlex -b, otherwise the window will not be freed and Ctrl-Break will not even work to stop NewsPlex.

-d dir Use directory dir as work directory for files, rather than the directory from which NewsPlex is started.
-e Enable insecure mode and allow remote connections. By default, only connections from the machine running NewsPlex are possible.
-f Delete unwanted articles. If this option is passed, the config/filters file is read on startup and all articles which match the patterns inside are deleted from the database. The format of the config/filters file is as follows: all lines starting with a # and all empty lines are ignored. Other lines must be composed of: a pattern for the From field, followed by a tabulation and by a pattern for the Message-ID field of articles to be deleted. Wildcards (* and ?) can be used in both patterns. Example file:
	# this is a comment
	annoying@annoying.com	<*@annoying.com>
	
	# this is another comment
	lamer*@lamers.com	<lamers*@*lamers.com>	
-g min-lines Enable asynchronous article retrieval for articles longer than min-lines lines and which seem to contain binaries. If min-lines is preceded with a minus, all articles longer than min-lines are retrieved asynchronously.
-h NewsPlex will print a short help about command-line options and exit.
-i Shutdown NewsPlex if the Internet connection is lost. This option works only if auto-updating is active (-u not specified).
-m Perform minimal exploration of news-servers and terminate. This command causes NewsPlex to run in a special Minimal mode, and only rate the news-servers listed in the config/servers file.

NewsPlex will try to connect to each of them, and retrieve an article head from each available news-group. If no article could be retrieved, the entire session will be logged into logs/unusable. Otherwise, the session log will stay in explore/server-name. No database or newsrc directories are used or created.

-n Renumber articles if news-server unreachable. This is the equivalent of the NPRENUMBER command. Specifying this option once causes the renumbering level to be set to 1, and specifying it twice causes the level to be set to 2.
-p port Use port port for establishing the NNTP news-server. By default, NewsPlex uses the standard port 119.
-r The list of news-servers is raw.
If this option is specified, NewsPlex expects a simpler format for the config/servers list of news-servers, without unique server-numbers in first column:

		server-url [flags] [# comment]
	
Entries with the same server-url are merged: each news-server is explored only once even if present multiple times in the list. It is possible to use the logs/results file as a raw config/servers file for the next exploration.
-s Do not establish an NNTP news-server.
-t Shorten file names to 8.3 conventions.
This options makes NewsPlex create files following the MS-DOS naming convention, so that NewsPlex can run on file-systems which do not support long filenames. It is not fully implemented yet, since in the database directory filenames still reflect the news-group names.
-u Do not auto-update the article database. It is still possible to explore a specific news-server by using the NPEXPLORESERVER command.
-x Do not use the XGTITLE NNTP protocol request for getting the news-groups list.
NewsPlex normally does not use XGTITLE for getting the list of news-groups matching a pattern. However, if the news-server does not support retrieving names of news-groups using wild-cards, then XGTITLE will be used. Since this command is often broken and does not return all the possible news-groups, the -x option allows to avoid using it. NewsPlex will instead get the full news-groups list and filter locally.
-w port Create a Web server on port port for controlling NewsPlex from a Web browser. The port value of 80 is the standard value you can use.
-z zombiedays Set the deleted article disappearance delay to zombiedays days. Value of 0 disables zombie creation. The default zombiedays is 7.
-A Read the config/authinfo file and set the login names and passwords as indicated in it. This avoids passing this information on the command line, where people can read it sometimes.
-D If this option is specified, NewsPlex will look at the dates of the newsrc files and defer the exploration of news-servers for which the date of their file is recent. If you restart NewsPlex after a short interruption, it is better to specify this option.
-I Assume that the Internet is always present. Use this option if NewsPlex is unable to correctly detect whether you are connected to the Internet at a given moment, and notably thinks that you are not while you really are.

Using this option will clutter the logs more easily with uninteresting host unreachable information, and also will cause the renumbering (see the NPRENUMBER command) to be performed when you try to request an article while not being connected to the Internet.

-M This option makes NewsPlex read the etc/retrievd file on startup and mark as retrieved (see section below) all the article descriptions whose Message-IDs are listed in this file.
-N This option makes NewsPlex always return an empty list of news-groups in response to the NEWGROUPS request. The default behavior is to return a full news-groups list.
-P login password This option makes NewsPlex require entering the specified login name and password before the extended commands (NPsomething) are available. By default, all commands are available. This option also protects the Web access.
-R Enable the marking of retrieved article descriptions in the article database (see section below). This option also enables the logging of the Message-IDs of retrieved articles into the etc/retrievd file.
-S Skip news-groups which appear as empty in the global news-group list when exploring news-servers. By default, NewsPlex tries to explore news-groups which appear as empty by other means, in order to find all the articles they might contain in spite of possible incoherencies in the information provided by the news-server, but this is not worth it most of the time.
-T count Explore count news-servers at once instead of default. By default, up to 8 news-servers are explored at once in normal mode, and 16 in minimal mode. This is suitable for PPP modem connections. Increasing this number can speed up things up if you have a fast Internet connection. Otherwise, it could make things worse because of Internet connection contention and resulting timeouts. Also, the more news-servers are explored at once, the more CPU and memory are used.
-U login password This option makes NewsPlex require entering the specified login name and password before any other commands can be entered. By default, the login and password are not required. This option also protects the Web access.

Security and access control

By default NewsPlex only allows logins from the machine on which it runs and all clients can do anything. This applies to both NNTP and Web accesses.

If passed the -e option, NewsPlex will allow NNTP and Web logins from any machine. This will completely remove any access protection.

It is possible to protect both NNTP and Web access with passwords. The passwords are the same for both accesses.

Two login and password couples can be used to separately protect user and supervisor (administrative) access.

Web access is protected with the supervisor login and password if a supervisor login and password are set, and with the user login and password if the user login and password are set and the supervisor login and password are not set. When the correct login and password are provided, all functions become available.

If the -P login password option is passed, supervisor NNTP access will be protected and only standard commands will be available to users which did not provide the correct values. People will notably not be able to see what news-servers are used by NewsPlex. However, since NewsPlex does not modify the headers of articles, the original news-server data will still appear in them, in the Path and Xref lines. If the -U login password option is passed, even normal user access will be protected and only the authentication command will be available until correct authentication data are provided. If -U is passed without an additional -P, every user will be priviledged as soon as it logs in. By passing both -U and -P, one can separately protect both normal access and supervisor access.

Adding and deleting news-servers

NewsPlex must be stopped for a news-server to be added to the config/servers file. Then it should be restarted.

To delete a news-server, for example because it has been unusable for a certain time, first delete the article descriptions which still point to it, using the NPDELETESERVER command, then delete or uncomment its entry in the config/servers file and restart NewsPlex.

Remembering retrieved articles

NewsPlex has a feature which can help you to avoid retrieving the same article twice if:

If you start NewsPlex with the -R option, NewsPlex will change the article description of an article in its database after this article has been successfully retrieved for the first time by a news-reader. It will add a "/r " prefix to the From field of the article description. If an article was previously from John Doe <john@doe.com> it will become an article from /r John Doe <john@doe.com>.

The articles themselves are not modified when they are actually retrieved.

If a news-reader later requests a full list of articles for a news-group, it will get the modified article descriptions. Normally, news-readers only request new article descriptions, so they do not see the modified article descriptions unless they request everything.

NewsPlex has a second way of remembering which articles have already been retrieved: it stores their Message-IDs into the etc/retrievd file. This is also enabled with the -R command-line option. It allows to keep the information even after the article has vanished from all news-servers where it was present, and hence the article description, containing the "/r " prefix has been deleted.

The -M command-line option makes NewsPlex read the etc/retrievd file on startup or restart and mark with "/r " all the articles whose Message-IDs are listed in this file.

This option is useful if you add a new news-server to the config/servers file and it has articles which you have already retrieved in the past, but which do not exist on any other news-server anymore. Except for the date, such articles would appear as new, because NewsPlex does not keep information about articles once they disappear from all news-servers. If the -M option is specified, NewsPlex will mark these old articles with "/r " and you will be able to avoid retrieving them again.

There is presently no way of doing the opposite, that is transferring the Message-IDs of previously retrieved articles from the article database into a file.

Error codes

If NewsPlex fails to start, it will print a diagnostic message. This message will also be logged to the journal file. NewsPlex will also return a specific error code. Presently, the following error codes are possible (C language syntax):

typedef enum {
    ERROR_OK,			/* No problem */
    ERROR_HELP,			/* Help was printed */
    ERROR_BAD_OPTION,		/* Bad option was passed */
    ERROR_NO_WORK_DIR,		/* Missing work directory */
    ERROR_NO_JOURNAL,		/* Could not open the journal file */
    ERROR_ANOTHER_SERVER,	/* Another server already running on port */
    ERROR_NO_CONFIG,		/* The config directory does not exist */
    ERROR_NO_AUTHINFO,		/* Option -A passed but no config/authinfo file */
    ERROR_BAD_AUTHINFO,		/* Incorrect config/authinfo file */
    ERROR_NO_GROUPS_YES,	/* Missing config/groups.yes file */
    ERROR_NO_LOGS,		/* Could not create the logs directory */
    ERROR_NO_ETC,		/* Could not create the etc directory */
    ERROR_NO_LIST,		/* Could not create the list directory */
    ERROR_NO_EXPLORE,		/* Could not create the explore directory */
    ERROR_NO_ASYNC,		/* Could not create the async directory */
    ERROR_NO_LONGNAMES,		/* -t not specified but disk is 8.3 only */
    ERROR_NO_NEWSRC,		/* Could not create the newsrc directory */
    ERROR_NO_DATABASE,		/* Could not create the database directory */
    ERROR_NO_UNUSABLE,		/* Could not create the logs/unusable file */
    ERROR_SERVERS,		/* Error in the config/servers file */
    ERROR_GROUPS_FILES,		/* Error when processing the groups files */
    ERROR_ASYNC,		/* Error when initializing async gets */
    ERROR_NNTPSERVER,		/* Could not establish an NNTP server */
    ERROR_ASYNC_THREADS,	/* Could not create async retrieval threads */
    ERROR_MAX			/* Dummy end of error list */
} eStartupErrors;

Installing NewsPlex on Windows desktop

You can create a shortcut for NewsPlex on the Windows desktop, passing the desired options in standard.

Drag the NewsPlex icon onto the desktop from the directory where the distribution has been unpacked; this creates a shortcut for the program. Select Properties from the shortcut's right-mouse-button menu, click the Shortcut tab, find the Target input box and add a blank and all the necessary options (at least -b probably, for background operation) at the end of the command associated with the shortcut.

In the Start in input box enter the name of the work directory where you created the config directory and the config/servers and config/groups.yes files. You can also pass the name of this directory with the -d command-line option.

From the Run list, select Minimized to avoid Windows switching to full-screen when starting NewsPlex (this happens on Windows 95).

Since version 2.8, the NewsPlex executable has its own icon, but if you do not like it, use the dialog box associated with the Change Icon button to change the icon to something else.

Finally, you can rename the shortcut into NewsPlex, either from the right mouse button menu, if the Rename option is available there, or by changing the name of the shortcut file to NewsPlex.lnk. The shortcut file is located in the Desktop subdirectory of the Windows installation directory on Windows 95. On Windows NT, it is located in the Desktop directory of your profile.

History of updates

Major improvements are in bold.

Version 2.8

Version 2.7

Version 2.6

Version 2.5

Version 2.4

Version 2.3

Version 2.2

Version 2.1

Version 2.0

Version 1.9

Version 1.8

Version 1.7

Version 1.6

Version 1.5

Version 1.4

Version 1.3