MV Communications: 1996 outages/events

1996 outages/events

Dec 29 1996: The T1 to Peterborough is out as of about 22:30. NYNEX has been contacted and we're awaiting a status. UPDATE: The link is back up at 6:30AM Dec 30 1996

Dec 7 thru Dec 10 1996

MV (along with much of NH) lost power late Saturday night. Power was finally restored on Tuesday night at about 4:30pm, and the systems were brought up by about 6pm. There will probably be increased contention for modems as people try to catch up on their Internet activities; please be mindful of this. Other repercussions are also likely.

Dec 6 1996

Keene site down. At about 15:50 there was a power outage in the Keene location due to the snowstorm. Power was restored around 19:15 but the circuit to Keene did not come back up. Circuit was finally brought up at about 21:10.

Dec 6 1996

Salem outage. At about 18:30 Salem lost power due to the snowstorm. Power was restored at about 19:30 and the Salem site came back up.

Nov 24 1996

This Sunday at 1pm we'll be adding memory to a router in Litchfield and moving the Sprintlink connection to that router. This may result in inability to reach the Internet for anywhere from 15 minutes to an hour.
(The operation was a success, downtime was about 40 minutes.)

Nov 3 1996

The system disk replacement (see below) was done at 1AM Sunday morning, finished about 2:20. There is some doubt that this was the needed fix: one of the news disk was found to be making some unusual (but familiar as bad) noises. If the news system proves to be unstable on Sunday, that disk will be replaced as well. As always, see mv.info.outages for more.

Nov 2 1996

quartz (aka news.mv.net) crashed a couple of times November 1 during the expiration process, and also on Oct 31 during the normal backup of the system disk. We suspect the system disk, and expect to swap in another one early Sunday AM. If there is further instability then this operation may have to be moved up into the daytime Saturday.

Oct 31 00:10

The recent problems with routing outages into and out of Sprintlink were solved. The problem was that the router on our side of the link that we installed on Sunday Oct 27 was found to have insufficient RAM for I/O buffers. The routers on each side of the T1 send keepalive packets to each other periodically; when the load on the T1 got sufficiently high, our router was too busy swapping memory around to deal with the keepalives, and the router at Sprint reset the link. At any rate, the replacement router needs to have a memory upgrade before it can be put back into service.

Oct 27 1pm

We made some configuration changes to the ethernet in Litchfield, a process that affected all routing to the Internet for all MV users. As per our original announcement we had expected this to only take about an hour. The main body of work actually took about 2-1/2 hours, and chasing down some routing problems and other bad behaviour went on through 6:30pm. A detailed followup will be posted to the mv.info.outages newsgroup this evening.

Oct 14/15 1996

bort, which is an SMTP and POP server, crashed at about 11:30 Monday night (October 14) due to a corrupted directory. It was repaired and was up at about 01:15.

Oct 11 1996

At about 9PM mv.mv.com became unreachable. Problem was diagnosed as a failed ethernet card; ethernet card was replaced and the system was back up at about 10:20. shell access was unavailable during this time, and some nameservice was also out of order.

Oct 5 1996

We'll be doing some rewiring and equipment moving at the Peterborough location on Saturday, Oct 5, starting at about 7pm. This will take down this facility for at least the first part of this activity, while the terminal server is being moved and while modems are being reconnected. Total downtime should be about 1 to 2 hours.
(it was 3 hours... about half the modems were back after 2 hours, the rest by 10pm)

Oct 4 1996

The Concord dialin modems were down today from about 12:00 noon to 14:15. A person at the facility (not an MV person) came in unbeknownst to us to remove a piece of their equipment from our ethernet, and left the ethernet broken. Service was restored when we went on site to investigate and found the trouble.

Sep 29 1996

mv.mv.com and zircon.mv.net (aka news2) will be taken down for maintenance Sunday Sep 29 at 1pm. Both systems should be back within an hour, probably much less. The outage of mv.mv.com will affect all shell users and some IP users.

Sep 16 1996

NYNEX is working on upgrading the Merrimack switch, moving some circuits over to the upgraded switch. This has been having a negative effect on some of our dialin lines, including taking some of them out of service and creating noise and crosstalk on other lines. NYNEX says that these problems may continue through Wednesday (the 18th).

Sep 10 01:30am

news.mv.net stopped taking network connections at about 1:30 but was otherwise fine (i.e., our monitoring software still regarded it as "up"). It was detected at about 4am and rebooted at about 4:25.

August 18 1996 1pm

sard.mv.net crashed on Sunday afternoon and was rebooted about 50 minutes later. sard has the WWW server and does some of the IP login authentication, so those services were unavailable during the outage (meaning that you couldn't access the web pages, and that IP users in some locations would get login failures). mv.mv.com and bort.mv.net were given a quick reboot at the same time.
Note: it's been reported that authentication failures were occuring for some time before the system crashed as well.

August 4, 1996 1pm

mv.mv.com was taken down as scheduled Sunday Aug 4 starting at 1pm for the installation of another disk drive. System was back up at 1:50. There was one glitch: mv provides login authentication for the IP accounts that log into Litchfield (that is, it checks the username and password when you log in). We had switched the authentication server to another system a few days ago in anticipation of this outage, but it apparently didn't take. So for the period when mv was down for maintenance (1pm - 1:50pm) no IP accounts could log into Litchfield. My apologies for any incoveniences caused.

Aug 1 19:33 - 19:48

The portmaster (the device that the modems attach to) in Dover crashed tonight at about 7:30. It had to be rebooted by hand (i.e. by going to the site) and was up again at about 9:45.

July 30, 1996 05:00

bort.mv.net crash: mail server stopped responding at 5AM, corrected by 9AM

July 13 1996 01:05

mv.mv.com crashed with a corrupted directory. Directory was repaired and system back up at 01:58. During the downtime IP users were not able to log in to Litchfield dialin numbers.

Overnight June 18/19, 1996

News server upgrade

June 14, 1996

MV hosts CU-SeeMe netcast of Olympic Torch Relay in Nashua, NH.

June 9, 1996

News server changing for shell accounts, and how to cope with this change.

June 9, 1996

Shell accounts on gold, granite and jade are moving

June 3, 1996 through June 9, 1996

Nameservice is changing
IP accounts: you may need to reconfigure the nameservers that you use.

June 9, 1996

Selected nameservers being shut down