Mars Maintenance - 17 July 2017 6pm

Network Network Notices
1

Mars highsite took a bit of a hit last night with the rain we had. Seems like one of the sectors (Mars1) has a cable short which is affecting the POE injector. Most of the Backbone links are still fine as they get power from the Toughswitch but all the devices connected to the Injector (Sectors and one Ubiquiti Backbone link) have severe packet loss on them.

Myself and @Trojan will be heading to Mars after work tomorrow to assess the damage and potentially fix the issues. @Gekido has donated a 24 port Gigabit POE injector to the site so if this turns out to be the problem we should be able to get it sorted out tomorrow.

Expect some downtime from 6pm till 7pm (roughly).

Fyi… @Ogon and @JellyBean

4 Likes
2

Mars trip diagnoses:

Faulty RB on Mars1 sector sent power back through the POE injector and fried an Ethernet port. This also hurt the Mainboard of the core router, even though we can still access it from the Wug, any device plugged into it has severe packet loss on ping or trace.

Mars1 and Mars5 sectors are offline.

Waiting on @Ogon to have a chat with @Committee about the replacement parts.

1: New RB for Mars1 sector
2: New Core Router
3: New POE Injector (I wouldnt risk plugging anything else into it.

As soon as we receive the replacement parts I will schedule a trip back up to get the site back to its former glory :slight_smile:

1 Like
3

Mars maintenance 90% complete, just some routing to finish up on the new core. Some pictures below for those who are interested:

New Mars1 Sector Basebox RB:

IMG_20170718_1815488.jpg3672×4896 2.91 MB

New RB3011 Core RB and New Gigabit POE Injector:

IMG_20170718_1935419.jpg3672×4896 2.77 MB

8 Likes
4

That looks good. What does the box in the middle do?

5

Top left: Power Supply, they implemented this PSU when Eskom was doing the load shedding thing.

Top Right: POE Injector which powers all five sectors.

Middle: Mikrotik 3011 Core Router which handles the Local OSPF and client BGP Routing protocols.

Bottom: 8 Port Toughswitch which powers and joins all the Backbone Links together.

2 Likes
6

I would also like to extend a big thank you to @Trojan and @quincy for assisting with the work, very much appreciated guys. A job well done :slight_smile:

2 Likes
7

Ah, so it’s the PSU for the injector.

Another thing if I may, why not use the RB3011 as the backbone switch? Would it be too much strain on the router?

Edit: I just realized that the Toughswitch has POE out.

1 Like
8

Its the PSU for the entire site, powers the Toughswitch as well as the POE injector.

1 Like
9

All but one, the firestarter link is ubnt link and therefore plugged straight into the core.

Thanks to @Firestarter, @Trojan and @quincy for the work you guys put in, to get the site up and running again.

4 Likes