Ok, so my son is 8 and that means he automatically likes Pokemon and minecraft.
He also likes anything with LED’s (geek in training I believe… blinky oooooh shiny…. sorry, distracted.)
So awhile back I came across this post BLE controlled Minecraft nightlight. I figured, this is perfect. So I started order some parts (making a few changes of my own along the way) and printed out the cube and base.
I then soldered up the led strips I had from a different project. Finished wiring everything up based on Mr. McMillan’s guide. One of the changes I made was I needed to make a different base piece. The cover originally designed required some larger feet than I had for it. So I designed a new piece.
Had an issue the weekend before christmas and had to order some replacements. So here I was Christmas Eve soldering everything up.
And I got it all working just in time. So Christmas morning it was sitting beside all the presents already lit up. I didn’t get a pic of it under the tree. Kind of wish I had. But here is a pic of it light up on my desk.
So last week my big storage box started acting up. Random reset, dropping a drive, all and all, not good.
So let me give you a quick rundown of this storage box. I am running freenas. I have a total of 11 drives currently. 9 of these drives are 2TB drives. Configured in 3 Raid 5 configurations. There is a small OS drive and a 128gig SSD just for cache. Then striped across giving me a total of 9.63TB of storage with redundancy. I store everything here, all my video and photo work. My media collection. My ESXi environment mounts iscsi off this thing. So it’s pretty critical my geek life.
I did all sorts of testing. Flashed the OS drive. Replaced the OS drive. No matter what I did. 4 minutes uptime, kernel panic and reboot.
So I ordered new parts which arrived yesterday. I take the system out of the rack, put it on the table, open it up…. found the problem…
Ouch. A small fire in my server.
So things went from bad to worse. Shortly after finding and fixing this. I reinstalled the OS and brought everything up for a 24 hour burn in. This worked. Ok good, lets go back to the SD card for the OS. Fresh install, 24 hour burn in. Lets go!!
12 hours in. System reboots. Doesn’t come online… No prob, Ill fix it when I get home…….. (do you see the foreshadowing here? cause I didn’t)
I get home, not booting right… Ok, reinstall os…. nope. ok, maybe sd card is bad. Back to the SSD. Nope..
Clean OS. No auto Import. Everything is fine…. import zfs volume…. kernel panic. Dead..
Time to research. Ok so from the inter-webs my prognoses is “screwed, data gone.
Apparently desktop memory and zfs are to blame here. Not like I wasn’t trying to keep my data. I had 3 raidz vdevs in a zfs pool.
So after contemplating all my poor poor data I decided to try to recover it.
Disk scans (SpinRite for 36 hours) = nothing
zdb scan (multiple hours but kept crashing because ran out of swap) = nothing
OpenIndiana live cd = nothing
Finally I found a post where someone talked about trying to force the volume only as read only. I figured, “hey, I’ve already spent 4 days trying to recover, why not”
So I boot up freenas. Get on the console and type
zpool import -f -o readonly=on -R /mnt vol
It didn’t kernel panic….. wait, what?!
Holy $%^&* IT MOUNTED!!! I’m jumping through directories all giddy that my data may still be intact. But read only isn’t going to do me much good. Need drives!!!!
I don’t have 10tb of external drive…. AJ!!!!
So I go to my buddies and steal all his externals. I plug the all in at once and start the very very very slow copy. After 5 days of copying to externals I was finally able to rebuild and start putting my data back.
So now the lessons learned:
Regularly check that your offsite back ups are working
Build a secondary nas for snapshot backups (this box will eventually be at AJ’s since we have a VPN between our places)
Identify what is replaceable and what isn’t and dump that somewhere else too.
This was a long process but its coming to a close. I will be doing snapshots of critical data to a secondary freenas box. Once the initial snapshot is done, I will take the box to AJ’s and the snapshots will continue to backup there.
So what that does it every time there is a keepalived state change it notifies that script. That script then runs additional scripts based on the state. So when it goes to “MASTER” or at boot time (which is why i have to put that sleep statement in there) it runs usb-connect.sh. Which just has a couple commands
Notice line 5, the priority is a higher number than in openhab1. This means that when the boxes communicate they will negotiate who gets to be master. Then either box can start openhab and whoever is running openhab gets the USB z-wave stick.
So lets do a brief summary of what these pieces of software do for me and what I currently have configured.
Openhab – “a vendor and technology agnostic open source automation software for your home.” That is what the site says openhab is and they are right. Openhab is a core home automation system that has been designed to work with MANY different vendors and systems to make your home smart. I have friends that got stuck with z-wave because they spent a bunch of money on z-wave and don’t want to replace it. But there is also belkin stuff, wifi modules, home built devices (this is a BIG problem for third party controllers.) Later on in this post you will see that I have used this agnostic approach to home automation to my benefit.
Mosquitto – “is an open source (BSD licensed) message broker that implements the MQ Telemetry Transport protocol versions 3.1 and 3.1.1.” Okay, not quite as cut and dry as openhab. So I’ll see if I can help. MQTT (MQ Telemetry Transport) is a system in which devices and services can connect to a central system and communicate via very small, very efficient messages back and forth. This efficiency helps in speed and bandwidth. A device can connect and just wait for commands, a service can send a message to the mqtt server which in turn immediately passes it to the device connected. MQTT is EXTREMELY more robust, but that is a simple paraphrase description of it.
MQTTWarn – “a pluggable MQTT notifier.” Hmm, not as helpful, but now that you know what mqtt is you probably understand this a little better. Jan-Piet Mens, the creator of mqttwarn and a very nice guy (I have personally had experience working with him in trying to accomplish various things) created this wonderful middleman piece of software. Its almost like grand central station, or your telephone switch board, or the traffic cop. Its a beautiful thing, I use this personally to do the following.
update dashboards in my office with current battery power on a couple devices using owntracks to get the battery data and pushing info to dashing dashboard.
Show current bandwidth usage from my router (python service I wrote to query snmp data from my router and publish via mqtt and then pushing info to dashing)
Pushing various alerts to prowl/growl
Push notices of events to my kodi installations
OwnTracks – “Your location companion.” OwnTracks is an application that can run on android and IOS devices that uses the internal GPS info and reports back to your mqtt server. This allows for a little better sense of privacy about our tracking info, but lets be honest if you have your phone on you, you are probably being tracked. But it doesn’t mean we have to give our info to everyone, so we use our own systems to track us. This also reports battery info with the location data it sends back to the system. Why would you want this? Well I use it to detect when I am home for presence detection. I also use it to have my home automation system know when I am leaving the office each day.
For devices I have integrated, I have..
a couple z-wave devices
2 phillips hue lights
2 belkin wemo switches
Logitech Media Server (squeezebox server) for media
So what am I doing moving this?!?! Well, this is a good question. The system does work in its current setup. However, as every project must, there has to be a very high WAF (Wife Acceptance Factor) if the project should ever be allowed to leave your workspace… Since I am pushing very hard to start putting some bigger pieces in the rest of the house (home built IR blasters, wall panels, in-ceiling speakers for voice notifications) I needed to increase the WAF. So, I have found that there are 2 things that make this much easier.
Must be easy to use. UI is key here, if others in the house can’t use it, its junk.
Must work. If the wife can’t turn on a light because something is down, its junk.
Those 2 key pieces of info really fit and help. So UI I believe I already have covered with a nice easy touch interface available on EVERY device in this house including some tangible remotes. So number 2 is the obstacle to tackle. It must work. So redundancy/fault tolerant is key. I have 2 seperate esxi environments in my setup here at home so I am going to place 2 openhab systems in virtual environment with heartbeat, failover and a virtual IP. To share the physical z-wave stick I am taking the raspberry pi and using virtualhere to share the 1 USB device with both controllers. This does still present a single point of failure for z-wave stuff. But it is not all the system so its better. If anyone has any thoughts on how to add redundancy to that Im all ears.
So here is what my goal is for my controller setup.
So you have seen my woes. I learned ALOT about 3d printers during the build and trials and tribulations. But once I had it pseudo working the first thing I wanted to print…. UPGRADES!!!
“#$%^& YES!!!! Print your own upgrades!” I yelled at the printer!
I printed a new fan shroud that has a vent to help cool the plastic as well as a slot for a 9 gram servo motor.. Why you may ask? Well, its for the auto bed level!
This seemed like the holy grail at the time, it is still pretty cool, but I have found it does put some extra wear and tear on your z-axis nuts and rods.
I basically build it based of this guys videos
This is still a pretty cool feature. I have even used the time it takes to auto level to change my heat bed temps by using some custom “start gcode” in CURA
G21 ;metric values
G90 ;absolute positioning
M82 ;set extruder to absolute mode
M107 ;start with the fan off
G28 ;move Z to min endstops
In short, my CURA profile has the heat bed set to heat up to 55 degrees. I personally like it at 65. So once that initial 55 is hit for the heatbed and the hotend it starts this custom G-Code. That first line says “set the head bed temp to 65” then it moves on to some of the other settings. The G28 and G29 are related to the auto bed leveling. So while its probing various points, the bed is heating up to my desired temp.
Its funny I have this little guy printed in black abs, black pla, red pla and gray pla in my house.
This hurt. For over a week I had my printer ripped back apart.
I will show a few more pics of the build and then go into some MAJOR things I learned. Some links to really good resources and some pics of various prints I have done.
This was the picture after I got a lot of the main support structure together. I thought I was getting close to being done. Oh how wrong I was..
All of this info so far happened in the time between Wednesday and Saturday that first week. So Saturday night the wife is off to work, the kids are pre-occupied. PRINTER TIME.
So I am putting on the heat bed the the y-carriage support (that acrylic plate in the middle.) You have springs you put over the screws so that it has constant pressure keep the plate level. I am pressing these things in…. crack!!
Yep, broke a piece of my printer before I even plugged in it.
I was devastated. However, not the end of the world. I continued on and just jammed a sticky note pad in there… don’t judge me. Desperate times called for desperate measures!
At that point I stepped away for 2 days and began working on it again going into the new week. Electronics time!!!
Now we have to hook up all that stuff.
Look how clean a RAMPS board looks before you hook everything up.
So I get everything hooked up. I’m done. Lets hook it to the computer, upload a test pattern for the motors and make it move.
Why isn’t it registering…. Why is that power light coming on and fading out…. Did I get a bum arduino mega? No worries!!! I have an extra one from another project, lets test it.
Out comes the magic smoke…. 1 of those little green stepper motor boards was on backwards and fried not only the arduino mega, but the mega, the RAMPS and 4 of 5 stepper motor drivers. Thank you amazon for having just that kit available for someone like me.
Now I am considering just setting it all on fire, molding it into something, and calling it “3d printer art” but we have it powering up now. So I had to give the dining room table back to the wife (she hates when I take over the table for projects)
So its time to start trying to print.
HA!!! I wish it was that easy. 1 simple word for anyone considering 3dprinting. CALIBRATION!!!! Do it. Do it again. Keep doing it.
That clicking sound in the video. Yeah turns out that is bad. I was having problems with the motor being able to push the filament through the nozzle.
Calibrate your temperatures! Make sure you aren’t too close to the bed. Sigh. This went on for awhile.
So much heat transfer into the extruder that it softened and jammed up. I cleared this SOOOO many times.
Look on the top of that heat block. See that little bit of leakage? That became the bane of my existence. Long prints and I would get a random build up of that plastic, it would drop on my print, cool, and snag the head.
Fixed the jamming with adjusting my cooling settings and print head temp.
Fixed the leaking head by pulling that throat tube out and putting some plumbers tape on it. Shouldn’t have needed it, maybe it was a bad tube, but worked like a champ since.
I had another problem later with all the vibrations my heating element fell out of the heat block. That was weird. Had to tighten everything down again.
Thermister broke… Replaced that.
Replaced nozzles a couple of times due to bad clogging (nozzles are cheap, order spares, you may be able to get acetone to dissolve a lot of the clogs, but who wants to wait multiple days)