New Firewall Build

So I have been running my home network on a home built router for well over a decade now.    Many moons ago when I lived in my apartment a client of mine was working on converting Eckerds to CVS stores.   As part of said conversion he had a ton of this little MaxTerm thin client machines.

3759951495_91064ca6e9

Which was not bad at all.  It has a little bit of memory, enough cpu to run as a router and an expansion slot.   Throw an extra network card in there and there you go.  Next to no power router.  I have been running http://m0n0.ch/wall/index.php since the beginning.  M0n0wall is now a dead project.  It will be missed, it sparked some of the great pieces I use today.   pfsense, freenas ect.   I have the last version on my little thin client and it runs well.

geek-is-coolBut like everything, eventually you run into something that makes you move on.  I had this recently.  Remember the great data apocalypse of 2015?  Well, that has sparked many changes in my backup replication plans.  One being that a buddy of mine and I are building secondary NAS systems for our freenas to put in each others server racks.  Yes, my buddy is a geek too.  So as part of this secondary nas setup.  We are going to be doing replication across the WAN.  Even though it uses SSH communication to do the replication.  We both felt doing an encrypted tunnel between locations was better.  Plus would let use gain access to each others resources without having to open unnecessary ports.  So we setup a nice encrypted tunnel.  Time to test some transfers…..

150kbs a second……..What-Meme-13

Now I never said I was a smart man.  But there is something wrong there.   AJ’s wan is 150/150 MEG!!!! (thank you FIOS) and mine is 50/50 also on FIOS!!!

No no no.  No sir, I do not want another.  I want the one I originally asked for!!!!!

Who broke that chair!!?
images

Ok.  I am calm now.  WTH is going on.   Checks the traffic graphs…

That’s not it.

Checks CPU graph on thin client….
hell-fire-1

Stops file transfer and checks again….
i_call_upon_the_power_of_ice_and_snow_by_fromzerotohero-d60aqs9

What the hell!?!  Now who broke THAT chair!!!!??

4565396395_c89b9c22f3_b

So it would seem that doing the encryption is causing a problem.  The cpu is just choking on the encrypt/decrypt over the tunnel.  That kind of sucks.  However we are also running on an old piece of hardware (12 years old?) and running on and EOL firmware.  Ok.  New Router time.

So this weekend I begin working on my new pfsense router.

Using this Intel D2500CCE Atom D2500 Dual LAN motherboard.

51mMt8cv6CL

Amazon

With a 2 gig memory chip this is ready to go.

I will post an update after I have this in and running for awhile and testing the encrypted transfers.  To see if it really is all in the cpu.

Green Sprinkler System Project (part 1)

So I don’t know why, but I have over the last year or so really gotten into doing things with renewable resources (mainly solar).   Well, lets say interested in.  I have a couple of solar panels I play with but nothing crazy.

So one day I get this crazy idea in my head.  I need a rain barrel.   But for idea_lightbulbwhat!?!?!  I had no real need for a Rain Barrel.  But this suddenly consumed me.  I NEEDED a rain barrel.  I checked craigslist and found an ad for someone getting rid of one.  Just needed to pick it up.   DONE!.
Now I had my barrel.  Now we need some water collection.   I looked around.  Rain collection can get expensive.  I saw collection systems over $100!!!!  That was crazy! That’s too much!!!

Much

 

I came across this Instructable – Rain Barrel and Diverter

This was right up my alley.   A couple bucks later I had a bunch of PVC and one saturday (when my step-dad was in town) we threw it together in no time.  Tested it out….  yes, I sprayed my waterhose on the roof to test it.  What’s it to you!?!?  pee_hose

So then when I regained my proper maturity level we waited for a storm.

 

Hahah b%^&*es I live in FL.  It rains like every 17 minutes during the summer.  So that night my son and I being the geniuses we are,  well, he was just following me, went outside with the flashlight to see how our water collection was doing.   Very surprisingly with just one section of gutter we were getting alot of water fast.   Before morning the barrel was full.

Here is what my setup currently looks like.  2015-05-30 18.08.34

 

 

 

 

 

This works surprisingly well.  However, I haven’t gotten to the point.  What am I doing with a 55 gallon drum of water.   Well, nothing…. yet.   Well, we used it a couple times as a sprinkler for the kids over the summer.

IMG_0603I built a little test rig to test out some sprinkler heads, and the kids and dog just ran around in it.   But it worked with my little pump I got from harbor freight.

 

 

image_11850This is the one I have. Water Pump.  I got this little pump specifically for a couple reasons.

  1.  Cheap.  This little guy was 40 bucks.
  2.  12v.   Why does that matter?  Because I want to power it off a 12v battery.

 

I’m sure at this point you are thinking “OK there guy.  So you have a bunch of water, you have a pump that can run off a battery and you have some sprinkler heads.   So what are you getting at…… ”

You hear that?  That was the click of it all coming together.  I am making a self sustaining green sprinkler system.   Rain Barrel to collect the water.   Battery charged from a solar panel to power the pump.  Easy right?  No.  No.  No.

Getting in to researching solar and batteries has forced me to start working on a charge controller circuit for the battery to avoid over charging or over discharging.   Why not buy it?   Harbor Freight has some.  You are right, but I am cheap and just buying stuff would make for this to be a really boring blog.   I would just upload receipts all day.  No spank you.  This is more fun!

IMG_0602So where am I in this whole ordeal?  Well, not as far along as I would like to be.  But I recently was able to test the actual pressure of my pump.  It claims a max of 50psi.  Which is close to the pressure to your house.   But I needed to see some numbers and here they are!  That’s actually higher than I expected considering how rough this pump sounds these days.  I tested a few times and always 30psi or higher.  This is actually really good news.  This means I should have a decent water pressure for my sprinkler heads.   I do not have a large front yard, but I am still looking at 5-6 heads to get the coverage.  In the next week or so I will be modifying my test rig to support 5 heads and see if I can run all 5 or if I need to split it into zones.  I am not opposed to doing the zones split, but it would be easier not to.

 

So that’s where I am.  The plans as I finish this are below.

  • add additional barrels
  • micro-controller to control when to turn on and off the pump
  • that same micro-controller will read a custom sensor I am building to determine water level to report back to my HA system.
  • HA system will handle the scheduling of watering using advanced rules.  ex: is rain expected in the next 12 hours?  has it rained in the last 12 hours?  If either of those, no need to water.   Is there enough water in the tank to run the pump?

 

 

Some projects and upgrades

So I have a few things on order coming from china for some various projects.

I figured since I haven’t posted much lately I will post some pre-project dribble.

Phoenix Connectors – Aliexpress

phoenix connectors I have these coming to be able to connect audio up to my ClearOne XAP800.   ClearOne XAP-800 Professional Audio Conferencing System Pic 4 xap800_l

The XAP800 is going to be added to my Home Automation system to be able to control/mix/route audio from various sources to various destinations.  If you want to read more about what this can do check out my buddy here.

http://hazymat.co.uk/2015/04/multi-room-audio-options/

 

3d Printer Upgrades

Its funny.   The whole time I have had my printer I have been constantly wanting to upgrade.  Maybe because I built my printer from a kit and know each piece.  I look at upgrades thinking, this will help just a tiny bit more.  This will make it a tiny bit better.  But also, I enjoy these projects.  So I have what I can only hope are my last big round of updates.  Nevermind.  I just thought of one more.  Bowden extruder…. but that’s for another post.

Motor Coupler – Aliexpress
The motor coupler is going be to used to connect my 5mm drive shaft to my below 8mm threaded rods.

Lead Screws w/ Copper Nut – Aliexpress
Replacing my current threaded rods and nuts with some nice thick Lead screws and longer copper nuts should add some additional stability and durability in that I won’t have to replace the nuts regularly like I do now.

Aluminum Build Plate – Aliexpress
The aluminum build plate will replace my glass bed.   I have hard it does well for heat distribution but also I need it for the below proximity switch.

Induction Proximity Switch – Amazon
I am going to get rid of current auto level setup.   I am going to the inductive sensor so there is no longer going to be a swinging arm.  This will hopefully improve start up time, and keep better accuracy.

shoppingUT82quJXalbXXagOFbXSReprap-Prusa-i3-3D-printer-parts-Anodized-Aluminum-BUILD-PLATE-for-Heated-Bed-3D-Printer-RepRap.jpg_640x64061fxf9nPCFL._SL1100_

 

Random

20 555 Timers – Aliexpress
ne555I have these 555 timers coming because I plan to use them on a small solar charge controller project I need for my Green Sprinkler system Project.

 

100 Optocouplers – Aliexpress
111781498246_1A couple weeks back I designed and tested a small circuit that will detect when the 12v accessory line in my car is turned on and off.  Using these optocouplers I will be able to use this signal to either power on my raspberry pi in my car, or to tell it to turn off.  This is going to be useful so that the pi doesn’t turn off as soon as I get home, but will be signaled to turn off in about an hour.  This will give enough time for sync jobs to run, backups and any config changes I want to do.

So there it is.  Some of my up coming projects

  • Big Printer upgrade
  • Solar Charge Controller
  • CarPi Power circuit deally (i need a better name)
  • Green Sprinkler System
  • Home Automation Audio Router

juggling-businessman-image

This guys has his shit way more together than I do.  Look at him!  He’s wearing a suit….  I barely wear pants!!!!

 

 

 

 

 

 

OpenHab crashing with Z-Wave FIX IT FIX IT FIX IT FIX IT!!

So,  my openhab system periodically decides to leave the building.  Appears there is a problem from time to time when the z-wave binding loses communication to the z-wave stick it gets upset and tells openhab to take a hike.

This is bad.  Once because it exposed something I missed in my fault tolerance.   I had compensated for network issues and full machine failover.  But the actual process going belly up…. ooops.  My Bad.

Soooo I see it crash while at the gym today and the only thing in my head….

gSHIj

So I appear to have done that.

Let me bring you up to speed on the current state of my home automation.  After the great NAS failing of 2015 I was forced to reduce some of my virtual environment.   I have not brought my secondary HA controller back online yet.  However, it appears that still using keepalived I am able to help address this random problem.

I have added in a new option in my keepalived.conf

 


vrrp_script chk_hahealth {
    script "/usr/local/sbin/healthcheck.sh"
    interval 10 # check every 10 seconds
    fall 2 # require 2 failures for KO
    rise 2 # require 2 successes for OK
}

vrrp_instance VI_1 {
   state MASTER
   interface eth0
   virtual_router_id 220
   priority 150
   notify /usr/local/sbin/notify-keepalived.sh
   advert_int 1
   authentication {
        auth_type PASS
        auth_pass fakepass
   }

   virtual_ipaddress {
      192.168.2.90
   }
   track_script {
     chk_hahealth
   }
}

So what this does is add a keepalived health check.   Every 10 seconds keepalived runs the script /usr/local/sbin/healthcheck.sh and gets an exit code of 0 or 1.  0 if all is good.  1 if the world fell apart.

Environmental concept. Some images in montage provided by NASA (http://visibleearth.nasa.gov/)

The code for this script is


#!/bin/sh
SERVICE=openhab;

if ps ax | grep -v grep | grep $SERVICE > /dev/null
then
 echo "$SERVICE service running, everything is fine"
 /usr/bin/logger "$SERVICE service running, everything is fine"
 exit 0
else
 echo "$SERVICE is not running"
 /usr/bin/logger "$SERVICE is not running"
 /etc/init.d/openhab restart
 exit 1
fi


Explanation:

So this script just checks to see if the openhab process is running.  If its good, exit 0.  If its not, exit 1 but go ahead and try to restart openhab.  When keepalived gets the exit 1 code it keeps track of it.  You will see in the config that there is a fall 2 line.  That means that if there are 2 exit 1 status’s keepalived will go into a failed state.  When the second HA box is back online this will force openhab to move over to the other one.  However, I have not seen this happen so far as openhab loads pretty quick so since there is 10 seconds between the checks the second check comes back with an exit 0 and resets the fall count.

 

 

 

 

MineCraft Nightlight Christmas Present

Ok, so my son is 8 and that means he automatically likes Pokemon and minecraft.
He also likes anything with LED’s (geek in training I believe… blinky oooooh shiny…. sorry, distracted.)

So awhile back I came across this post BLE controlled Minecraft nightlight.  I figured, this is perfect.  So I started order some parts (making a few changes of my own along the way) and printed out the cube and base.

Parts:
Adafruit Pro Trinket – 3V 12MHz
Adafruit Neo Pixel Strip
Capacitor
3d Printer
Adafruit BlueFruit LE Bluetooth module

 

2015-12-16 17.56.55

 

2015-12-16 17.57.19 2015-12-16 17.57.23

I then soldered up the led strips I had from a different project.  Finished wiring everything up based on Mr. McMillan’s guide.  One of the changes I made was I needed to make a different base piece.   The cover originally designed required some larger feet than I had for it.  So I designed a new piece.

Here is the alternate piece.  Thingiverse

2015-12-16 21.00.042015-12-16 21.00.012015-12-16 20.59.592015-12-16 20.23.34

Had an issue the weekend before christmas and had to order some replacements.  So here I was Christmas Eve soldering everything up.

2015-12-24 16.57.26

And I got it all working just in time.  So Christmas morning it was sitting beside all the presents already lit up.  I didn’t get a pic of it under the tree.  Kind of wish I had.  But here is a pic of it light up on my desk.

2015-12-24 17.50.24


Storage Problems (UPDATED!!!)

So last week my big storage box started acting up.  Random reset, dropping a drive, all and all, not good.

So let me give you a quick rundown of this storage box.  I am running freenas.  I have a total of 11 drives currently.  9 of these drives are 2TB drives.   Configured in 3 Raid 5 configurations.   There is a small OS drive and a 128gig SSD just for cache.  Then striped across giving me a total of 9.63TB of storage with redundancy.   I store everything here, all my video and photo work.  My media collection.  My ESXi environment mounts iscsi off this thing.  So it’s pretty critical my geek life.

I did all sorts of testing.  Flashed the OS drive.  Replaced the OS drive.  No matter what I did.  4 minutes uptime, kernel panic and reboot.

So I ordered new parts which arrived yesterday.  I take the system out of the rack, put it on the table, open it up…. found the problem…

IMG_0398

Ouch.  A small fire in my server.

 

Update!!!! (12/23/15)
So things went from bad to worse.  Shortly after finding and fixing this.  I reinstalled the OS and brought everything up for a 24 hour burn in.  This worked.  Ok good, lets go back to the SD card for the OS.  Fresh install, 24 hour burn in.  Lets go!!

12 hours in.  System reboots.  Doesn’t come online… No prob, Ill fix it when I get home…….. (do you see the foreshadowing here?  cause I didn’t)

I get home, not booting right…  Ok,  reinstall os…. nope.   ok, maybe sd card is bad.  Back to the SSD.   Nope..

Uhhhh WTF!?!?!

Clean OS.  No auto Import.  Everything is fine…. import zfs volume…. kernel panic.  Dead..

1234931504682

Time to research.   Ok so from the inter-webs my prognoses is “screwed, data gone.

original

Apparently desktop memory and zfs are to blame here.   Not like I wasn’t trying to keep my data.  I had 3 raidz vdevs in a zfs pool.

So after contemplating all my poor poor data I decided to try to recover it.

Disk scans (SpinRite for 36 hours)  = nothing
zdb scan (multiple hours but kept crashing because ran out of swap) = nothing
OpenIndiana live cd = nothing

Finally I found a post where someone talked about trying to force the volume only as read only.  I figured, “hey, I’ve already spent 4 days trying to recover, why not”

So I boot up freenas.  Get on the console and type


zpool import -f -o readonly=on -R /mnt vol

It didn’t kernel panic….. wait, what?!

Holy $%^&*    IT MOUNTED!!! I’m jumping through directories all giddy that my data may still be intact.   But read only isn’t going to do me much good.  Need drives!!!!

I don’t have 10tb of external drive…. AJ!!!!

So I go to my buddies and steal all his externals.  I plug the all in at once and start the very very very slow copy.   After 5 days of copying to externals I was finally able to rebuild and start putting my data back.

So now the lessons learned:

  1. Regularly check that your offsite back ups are working
  2. Build a secondary nas for snapshot backups (this box will eventually be at AJ’s since we have a VPN between our places)
  3. Identify what is replaceable and what isn’t and dump that somewhere else too.

This was a long process but its coming to a close.  I will be doing snapshots of critical data to a secondary freenas box.   Once the initial snapshot is done, I will take the box to AJ’s and the snapshots will continue to backup there.

 

 

Home Automation Quick Update

o since I made the home automation system failover its been great!…. except I kind of would like to know which box its running on it.  So I made a quick change.  A new Item on my dash board

String  Server "Server [%s]" {exec="<[/bin/cat@@/etc/hostname:60000:]"}

I have that in my items file.  Then in my sitemap I added

Text label="Currently Running on [%s]" item=Server

I have that at the bottom of my sitemap.

image

Home Automation Move (part 2)

So, with a friday and a saturday worth of work on my home automation move here is what is complete.

  1. install server os on vm
  2. install openhab and all bindings currently in use
  3. move openhab configs over to new vm
  4. shutdown old openhab-pi
  5. configure raspberry pi with virtualhere server to share usb
  6. configure new vm server to connect to raspberry pi to communicate with z-wave stick
  7. install keepalived on new server
  8. configure virtual ip as my new primary ip for openhab access
  9. configure scripts to run to start openhab and connect to shared usb
  10. clone server to secondary vm for failover
  11. reconfigure keepalived to make second box slave
  12. test failover

So let me show you my keepalived settings and my scripts.

/etc/keepalived/keepalived.conf

vrrp_instance VI_1 {
     state MASTER
     interface eth0
     virtual_router_id 220
     priority 150
     notify /usr/local/sbin/notify-keepalived.sh
     advert_int 1
     authentication {
           auth_type PASS
           auth_pass fakepass
     }
     virtual_ipaddress {
         192.168.2.90
     }
}

 

See the “notify” line?    That script is pretty simple.

#!/bin/bash
TYPE=$1
NAME=$2
STATE=$3
case $STATE in
        "MASTER") sleep 30
                  /usr/local/sbin/usb-connect.sh
                  /usr/sbin/service openhab start;;
        "BACKUP") /usr/sbin/service openhab stop
                  /usr/local/sbin/usb-disconnect.sh;;
        "FAULT")  /usr/sbin/service openhab stop
                  /usr/local/sbin/usb-disconnect.sh
                  exit 0
                  ;;
        *)        /sbin/logger "unknown state"
                  exit 1
                  ;;
esac

 

So what that does it every time there is a keepalived state change it notifies that script.   That script then runs additional scripts based on the state.   So when it goes to “MASTER” or at boot time (which is why i have to put that sleep statement in there) it runs usb-connect.sh.  Which just has a couple commands

/sbin/vhclient &
sleep 10
/sbin/vhclient -t "USE,4294967409"

I’ll walk you through this one.

  1. runs the virtualhere usb client.
  2. waits a few seconds so the client can detect shared usb on the network
  3. sends a command to the running client “-t = command” specifying to “USE” the device with the id “4294967409”

The usb-disconnect.sh is a single line!

pkill vhclient

Thats it.  Just shutdown the client.   So now when the box boots up openhab1 becomes “MASTER” for openhab.  It then executes the scripts to connect to the shared USB.  Then starts openhab.

Once the second box is in place, all the same scripts and all will be put in place with 1 single change.

 

vrrp_instance VI_1 {
     state MASTER
     interface eth0
     virtual_router_id 220
     priority 200
     notify /usr/local/sbin/notify-keepalived.sh
     advert_int 1
     authentication {
           auth_type PASS
           auth_pass fakepass
     }
     virtual_ipaddress {
         192.168.2.90
     }
}

Notice line 5, the priority is a higher number than in openhab1.  This means that when the boxes communicate they will negotiate who gets to be master.   Then either box can start openhab and whoever is running openhab gets the USB z-wave stick.

Home Automation Move (part 1)

I have been running openhab for over a year now on a raspberry pi. I also run mosquitto mqtt broker with a great piece of software called mqttwarn.   On my phone I run owntracks.

FullSizeRender

So lets do a brief summary of what these pieces of software do for me and what I currently have configured.

Openhab – “a vendor and technology agnostic open source automation software for your home.”   That is what the site says openhab is and they are right.   Openhab is a core home automation system that has been designed to work with MANY different vendors and systems to make your home smart.   I have friends that got stuck with z-wave because they spent a bunch of money on z-wave and don’t want to replace it.  But there is also belkin stuff, wifi modules, home built devices (this is a BIG problem for third party controllers.)   Later on in this post you will see that I have used this agnostic approach to home automation to my benefit.

Mosquitto – “is an open source (BSD licensed) message broker that implements the MQ Telemetry Transport protocol versions 3.1 and 3.1.1.”  Okay, not quite as cut and dry as openhab.  So I’ll see if I can help.   MQTT (MQ Telemetry Transport) is a system in which devices and services can connect to a central system and communicate via very small, very efficient messages back and forth.  This efficiency helps in speed and bandwidth.  A device can connect and just wait for commands, a service can send a message to the mqtt server which in turn immediately passes it to the device connected.  MQTT is EXTREMELY more robust, but that is a simple paraphrase description of it.

MQTTWarn – “a pluggable MQTT notifier.”   Hmm, not as helpful, but now that you know what mqtt is you probably understand this a little better.   Jan-Piet Mens, the creator of mqttwarn and a very nice guy (I have personally had experience working with him in trying to accomplish various things) created this wonderful middleman piece of software.  Its almost like grand central station, or your telephone switch board, or the traffic cop.   Its a beautiful thing, I use this personally to do the following.

  • update dashboards in my office with current battery power on a couple devices using owntracks to get the battery data and pushing info to dashing dashboard. 2015-11-13 16_53_22-My super sweet dashboard
  • Show current bandwidth usage from my router (python service I wrote to query snmp data from my router and publish via mqtt and then pushing info to dashing)
    bandwidth
  • Pushing various alerts to prowl/growl
  • Push notices of events to my kodi installations

OwnTracks – “Your location companion.”   OwnTracks is an application that can run on android and IOS devices that uses the internal GPS info and reports back to your mqtt server.  This allows for a little better sense of privacy about our tracking info, but lets be honest if you have your phone on you, you are probably being tracked.  But it doesn’t mean we have to give our info to everyone, so we use our own systems to track us.  This also reports battery info with the location data it sends back to the system.  Why would you want this?  Well I use it to detect when I am home for presence detection.  I also use it to have my home automation system know when I am leaving the office each day.

 

For devices I have integrated, I have..

  • a couple z-wave devices
  • 2 phillips hue lights
  • 2 belkin wemo switches
  • Logitech Media Server (squeezebox server) for media
  • 2 Max2Play raspberry pi setups.
  • 3 Kodi installs
  • 2 mobile devices via owntracks

 

So what am I doing moving this?!?!   Well, this is a good question.  The system does work in its current setup.  However, as every project must, there has to be a very high WAF (Wife Acceptance Factor) if the project should ever be allowed to leave your workspace…  Since I am pushing very hard to start putting some bigger pieces in the rest of the house (home built IR blasters, wall panels, in-ceiling speakers for voice notifications) I needed to increase the WAF.  So, I have found that there are 2 things that make this much easier.

  1. Must be easy to use.  UI is key here, if others in the house can’t use it, its junk.
  2. Must work.   If the wife can’t turn on a light because something is down, its junk.

Those 2 key pieces of info really fit and help.  So UI I believe I already have covered with a nice easy touch interface available on EVERY device in this house including some tangible remotes.   So number 2 is the obstacle to tackle.  It must work.  So redundancy/fault tolerant is key.  I have 2 seperate esxi environments in my setup here at home so I am going to place 2 openhab systems in virtual environment with heartbeat, failover and a virtual IP.  To share the physical z-wave stick I am taking the raspberry pi and using virtualhere to share the 1 USB device with both controllers.  This does still present a single point of failure for z-wave stuff.  But it is not all the system so its better.  If anyone has any thoughts on how to add redundancy to that Im all ears.

So here is what my goal is for my controller setup.

Openhab Layout - New Page (1)

 

My first Thingiverse Item (Light Saber Stand)

So I owed a buddy of mine a stand for a light saber.  I took his saber and measured it up and actually designed my first real 3d design.

Its nothing crazy and will have a couple more tweaks as the tolerances are a little too tight right now.

http://www.thingiverse.com/thing:1124780

My profile on Thingiverse is https://www.thingiverse.com/CrankyCoder/

terrific_duup_1_preview_featuredIMG_0332IMG_0333IMG_0334