Title Image

Network Migration COMPLETE

So, after my POST SKYRIM COMA, I finally got around to finishing my network migration. The old setup, if you remember, and the proposed new setup, were documented here. Now, the new state is done and executed HERE:

2013-02-06_18-16-11_750


The important thing is that all cables were custom cut to be optimal length, meaning no link from the server to the gateway is more than 3 feet. Also, EVERY link between the server and the world is gigabit. LAN transfers of backups went from 10MB/s to 20MB/s.

What will this mean? Not much. Maybe if 100 people were on, lag would be cut. This is more just an interest story :D

New Internet

from 10-12 PST on today, 11/28/12, the server will likely be down as my internet is upgraded to business class. Check here for updates; spread the word.

UPDATE: the dude is still working now at about 2PM pst...my ssh connection just died, so I assume we're down?

UPDATE 2: it'll take a bit for the dns to propogate...try the following as the server address:

  • surfrock66.yourhda.com:25565
  • mc.teh3l3m3nts.com:25565
  • 71.197.114.191:25565

Update 3: Well there's sort of a sucky deal...the new business class device isn't just a modem, it's a modem and a router, a router with an overly simplistic configuration. Allow me to explain via whiteboard pictures:

NEW SERVER TESTING

WE ARE MOVING NOW! USE THE TEMPORARY ADDRESS FOR A WHILE! UPGRADE TO 1.4.2

NEWSFLASH! We got a development build of Bukkit 1.4.2, so if you updated, log into the new server!

Ok! The new server is up and running! We are NOT go-live yet, we need more testing. EVERYTHING ON THE OLD SERVER STAYS THE SAME UNTIL GO-LIVE! I need to know if the new server lags, when it lags, how much tnt blowing up it can survive, etc. I need as many ppl on doing as much as they can at many times. TEMPORARY SERVER ADDRESS: surfrock66.yourhda.com

For that end...it's a backup of our world and you can abuse it however you want. Please though, keep 2 things in mind from me:

  1. This is a test, and the 2 servers will be merged likely next weekend, so don't spend a ton of time getting used to different worlds, we're not splitting the community or anything
  2. I will likely upgrade the test server to 1.4.2 and use a dev bukkit, as a test.
  3. Be careful which server you log in to! Don't log into the real server and blow things up thinking it's this one!
  4. When we go live...I'll update the MOTD with a "NO TNT!" order.

So, few other things/known bugs!

  • There is no dynmap right now, but when I get it set up, the temporary address will be "surfrock66.yourhda.com:50723"
  • The web irc client won't work right now with the new server, but you can connect with another client at address "surfrock66.yourhda.com" at port 50730 to channel #minecraft
  • Stats and blocklogging are dead. We may have an issue...I can't connect to my db but that may be a result of a few things, I'll work on that.

Please report any problems here, I need to know so I can fix before we go live! It's useful info to say "we had an issue with it lagging with 5 people on at XXXYYY time on XXXYYY date!

ROADMAP

  1. Repeated and ongoing stress testing of server and network.
  2. DONE: Implement a test upgrade to 1.4.2 on the new server, with a development build of bukkit - On target for Thursday/Friday
  3. 12/12 DONE: Implement a test upgrade of all of our plugins - On target for Thursday/Friday
  4. DONE: Test MySQL connection for BlockLog and BeardStats, which involves taking the old server down temporarily (as we believe it has a lock on writes to the DB). On track for potentially early Saturday morning
  5. DONE: Re-implement changes to the permissions nodes that I have been super delinquent on updating...ideally after all plugins are updated. On track for Friday or Saturday.
  6. DONE: Update Noob World by copying the dome to fresh chunks and implementing spawn protect - On track for anytime after the 1.4.2 dev build and worldedit are ready to go. WHEN TESTING WAS COMPLETE, I ALSO SUCCESSFULLY TESTED THE AUTO-RESET NOOB-WORLD SCRIPT.
  7. ROLLOUT! This involves STOPPING the old server, performing a full backup, restarting the old server (which at this point is griefable), transferring the backups to the new hardware (likely 10GB) which will take hours, then unzipping old files and initiating the new server with a DON'T GRIEF HERE message. Once it's live and connectable, initiate the DNS move so "mc.teh3l3m3nts.com" points to the new server, then wait 24 hours and discontinue the old server. ON TRACK FOR LATE SUNDAY!

TESTING UPDATE 1! As much as it's fun, blowing everything up isn't very scientific. For example...at legion base on the test server, there's a loop of sponge-water-lava-tnt that is just causing death. If someone is at legion base, the world crashes. I'm restoring a backup now. HOWEVER it did inspire me to construct a more controlled test which MadBrains saw...I used worldedit at the cheatarena2 to set off a sphere of tnt at every radius 1-15, and both servers performed about the same in-game. Here's the logs from the tests to see rough timecodes.

Old Server Logs Of The Test:

Spoiler Inside SelectShow

New Server Logs Of The Test:

Spoiler Inside SelectShow

TESTING UPDATE 2! Plugin testing is underway, and I was tracking it in a comment, but that was dumb. Here's the actual plugin status:

Spoiler Inside SelectShow

TESTING UPDATE 3! Major progress tonight. Worked out details of worldguard/worldedit/factions discussion and put it to administrators. Set up the new noob world, created its backup, reconfigured all the multiverse crap. Tested auto-restore script for noob world...works great. Also restored the super-old unused worlds for old times sake. Also sent a message to my host to hopefully work out the mysql problems with blocklog and beardstat. I tested the incremental backup plugins on the new server...working great. Finally, I actually had a real downtime that I was able to test the monitoring scripts I wrote...work like a charm. I would say we are completely ahead of schedule.

Owner’s Corner: Monitoring Script

We've never had a great monitoring solution. If something goes really wrong, usually I wait for someone to text me if I'm not there to deal with it. I've long wanted to initiate an automatic notification of downtime, and tried to use something like nagios to do the monitoring, but it ended up being overkill for this monitoring of 1 service on 1 server. So I decided to write my own solution.

I believe there may be some logical complications here, so I wanna talk out my logic and ask you, the readers, to poke holes in my thinking. This script isn't done yet, by far.

Read more »

Owner’s Corner: THE NEW SERVER! CONTINUALLY UPDATED POST!

IT'S FINALLY COMING! THE SERVER IS SCHEDULED TO BE DELIVERED TODAY! This will be an ongoing chronicle, full of pictures and such, showing what we're doing to ready this thing for production. It will not be a quick process.

Read more »