RethinkDB Gems

After idling in the RethinkDB freenode IRC chat, I think I'm going to start a collection of very good Q/A that I see:

<dubcanada> One more question, is there an easy way to duplicate all the content in the DB
<dubcanada> I'm trying to fill it up with a ton of records for testing but it seems to be taking forever lol
<AtnNn> dubcanada: you can do something like r.table('foo').insert(r.table('foo').without('id'))

Search for documents without field:

<nicklovescode> what's the best way to search for documents without a field? I tried .filter({ foo: undefined }) but that got an error
<srh_> nicklovescode: you can use .filter(r.row.hasFields('foo').not())

How to implement failover:

20:20 <bg__> i also have a question regarding failovers
20:22 <bg__> until #223 is resolved, is there a suggestion regarding how I should handle failovers? Meaning, if i have services that are using my rethinkdb, and I have tables that have replicas, how can I ensure I'm not trafficing requests to an unresponsive node?
20:23 <neumino> You need to have a connection opened to the server
20:23 <neumino> If the connection is dropped
20:23 <neumino> You can try to connect to another server
20:23 <bg__> okay. so lets say i have web1 and web2 which are nodejs web servers
20:24 <bg__> and they handle requests that require them connecting to rethinkdb and reading from one table
20:24 <bg__> and because im afraid of one serving dying in the middle of the night, i have two rethinkdb servers, rethink1 and rethink2
20:25 <bg__> what connection string do I use to have web1 and web2 connect to rethink1 and rethink2 that protects me if rethink1 or rethink 2 goes down
20:25 <bg__> i only have web1 and rethink1 right now. web1 explicitly connects to rethink1
20:25 <bg__> but I'm reaching a point where that is not safe
20:26 <neumino> Hum
20:26 <bg__> did that make sense?
20:27 <neumino> bg__, if the master of your table fails, you won't be able to write until you declare the master dead
20:27 <neumino> We currently don't have a nice way to do it, but the coreos folks did it that way:
20:27 <neumino> They fetch `http://server:8080/ajax`
20:27 <neumino> They fetch `http://server:8080/ajax/issues`
20:27 <neumino> If there is a dead machine
20:28 <neumino> They start a script that is going to kill it using `rethinkdb admin`
20:29 <bg__> ah ha.
20:29 <bg__> i did not know about those two end points
20:30 <bg__> i was going to have a script that polled the server
20:30 <bg__> i just wasnt sure what endpoint to use
20:30 <bg__> those seem to do the trick
20:31 <bg__> thanks@!
20:31 <bg__> is there any documentation about that?
20:32 <neumino> Hum not really :/
20:32 <neumino> If you start `rethinkdb admin`
20:32 <neumino> And then send `help`, you should get the docs for `rethinkdb admin`
20:32 <neumino> For `ajax/issues`, there is no doc for now
20:33 <neumino> Try to kill a server and you will see the error you get
20:33 <bg__> i meant /ajax and /ajax/issues
20:33 <neumino> It's something like "DEAD_MACHINE" I think
20:33 <neumino> Oh
20:33 <bg__> both seem very important to someone trying to use rethinkdb as a distributed system
20:33 <bg__> getting around this isn't difficult now that i know about those
20:42 <bg__> just out of curiousity, when you say 'the way the coreos folks did it' what do you mean?
20:45 <neumino> Oh, the engineers at coreos needed to set up a kind of failover with rethinkdb
20:45 <neumino> and that's what they did
20:45 <neumino> It's kind of the only solution now ^^
20:45 <neumino> (well you could skip `rethinkdb admin` and craft post request like the web interface do
20:46 <bg__> oh, got it.
20:46 <bg__> its a great solution for now.

Convert Unix Endings to DOS/Windows using Cygwin/TCSH

Small note for me, which converts all *.cs files recursively in a directory:

find ./**/*.cs -type f -exec unix2dos {} \;

Happy coding,
Brian Chavez

Visual Studio 2013 Loading Splash Screen Slow

I recently experienced Visual Studio 2013 loading very slowly at the splash screen. It was noticeable since I'm on a very fast dev box and accustomed to VS 2013 loading very fast (normally under 10 seconds).

So, something defiantly changed.

Problem turns out that I had recently opened a project under Git source control located on a remote share.

The remote share was no longer available, yet, the Team Foundation Git Source Control provider was still probing the network share which took up a significant part of the loading time. I suppose the Git provider was waiting for a network timeout.

Anyway, to resolve the issue simply remove the offending network shares under Git source control located here:

HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\12.0\TeamFoundation\GitSourceControl\Repositories

Under the Repositories key, you'll find a stash of GUIDs for each project the Git provider keeps track of. Locate the offending GUID keys that contain the network shares you want to remove and remove the entire GUID entry (along with the contents) -- and all should be well.

Happy coding,
Brian Chavez

@functions "This is not a valid location for a breakpoint" in Visual Studio 2013

Working with Visual Studio 2013 and I tried to set a breakpoint in "@fucntions" bout got back an error message that read:

This is not a valid location for a breakpoint.

image

That's funny, because it works in Visual Studio 2012. See:

image

Oh well, as a work around, move the @functions block up to the beginning of the cshtml razor view, and you should be able to once again set breakpoints:

image

Why? Who knows. :/

-Brian

Installing Freelan on Windows Server 2012

So, I was trying out Windows Server 2012. I tried installing freelan (http://www.freelan.org) as a service. If you know me, I'm a big fan of freelan, but for some reason, after every cold boot, there was no connectivity to the node over VPN. Specifically, this problem was only occurring on Windows Server 2012.

Generally speaking, I think it has to do with the freelan acquiring the OpenVPN TAP adapter. Changes in the operating system driver infrastructure in Windows Server 2012 somehow goofs up the freelan application as it tries to initialize itself and the driver.

Turns out, you need to force the TAP adapter to a connection state of "Always Connected". Somehow, when the TAP adapter is set to "Always Connected", Windows then makes the TAP adapter available to the OS networking infrastructure. So when freelan tries to initialize it, it's available for use. Network problem solved.

Here's how to do it. First, open the TAP adapter status, then click Properties.

image

Next, click Configure.

image

Set the Media Status to Always Connected.

This probably has a lot to do with people generally using OpenVPN's TAP adapter on Windows 8 or 8.1 too. Or, maybe freelan is just packaged with an old out-dated OpenVPN TAP adapter.

https://forums.openvpn.net/topic12865.html

https://forums.openvpn.net/topic10013-15.html

Clicking Outlook links open Blank/Empty Page in Chrome

Quick Note

google-chrome-about-crash Under the following conditions:

  • Google Chrome set as the default browser.
  • Microsoft Outlook running in "Windows XP" compatibility mode.

Clicking on links inside emails (or any hyperlink even inside contacts), Google Chrome will open up in a blank "Untitled" tab. UI menus and the ominibox are completely unresponsive.

Turns out, running Microsoft Outlook in "Windows XP" compatibility mode will cause Google Chrome to fail to open links in Outlook.

So, disable "compatibility mode" and all should be well.

image

Hope that helps,
Brian Chavez

Cleaning up Network Connections in Windows 7

image Windows makes a mess of all the network adapters when installing NIC cards. Sometimes, you remove network adapters, and you're left with something like "Local Area Network 3" that can't be renamed back to "Local Area Network". The problem is, Windows remembers the previous adapter (even if it is physically removed), so you can't rename the adapter, otherwise it would conflict with previously named adapters. View & remove all previously (non-connected) NIC adapters:

set DEVMGR_SHOW_NONPRESENT_DEVICES=1
devmgmt.msc

View > Show Hidden Devices

Now, you can remove any adapters that were previously installed (but no longer present) in windows.

After doing so, you should now be able to rename the network adapters back to "Local Area Connection".

Extra Tip 1: Using the "Network and Sharing Center", click on the icon to change the icon and network name.

Extra Tip 2: Start "Local Security Policy". Go to "Network List Manager Polices", and you'll be able to assign profiles to these networks.

Extra Tip 3: Click on an "Unidentified network"s icon, and click "Merge or delete network locations" to delete any previously remembered "networks".

Hope that helps,
Brian Chavez

Setup latest Redis on Ubuntu

image Setting up latest Redis on Ubuntu

  • add-apt-repository ppa:rwky/redis
  • apt-get update
  • apt-get install redis
  • nano /etc/redis/redis.conf
  • service redis-server restart

Extra: Ensure Durable Redis

(make Redis durable on every dataset change)

  • nano /etc/redis/redis.conf
  • Comment out:
    # save 900 1
    # save 300 10
    # save 60 10000
  • Change:
    appendonly yes
    appendfsync always
  • Ensure:
    no-appendfsync-on-rewrite no
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb

Cleanup: delete /var/lib/redis/dump.rdb since we're not using snapshots.

The AOF file is in /var/lib/redis/appendonly.aof; this is your dataset on disk.

Log files are in /var/log/redis/redis.log

Hope that helps,
Brian Chavez

HTTPie doesn't honor --style colors in mintty/cygwin

Be sure to check our $TERM variable.

cowboy@bigcowboy:~$ echo $TERM
xterm-256color

It should have the words "256color" before you can use httpie's --styles.

Specifically, the offending code is in httpie models.py: Line 25

colors = 256 if '256color' in os.environ.get('TERM', '') else 88

image

Hope that helps,
Brian Chavez

Compiling Kernel Modules for Raspberry Pi

image I tried loading rpi-pwm on my Raspberry Pi, but ran into some issues. After some tinkering I was able to compile rpi-pwm.c on the Raspberry Pi, but I kept running into Invalid Format issues anytime I tried to insmod rpi-pwm.ko which had the following dmesg:

rpi_pwm: disagrees about version of symbol module_layout

I guess this generally means I need to compile my module in the same way the kernel is compiled on the Raspberry Pi. Otherwise, the kernel will reject the module.

I've found that cross-compiling works best. This means, building your kernel and modules on a different platform rather than on the Raspberry Pi hardware directly. I did NOT have any luck compiling my kernel module directly on Raspberry Pi hardware.

In detail, first you'll want to cross-compile the Raspberry Pi Linux kernel so it builds Module.symvars and other intermediates. You won't actually use the kernel that is being cross-compiled, just the intermediates to compile your module. Next, you'll cross-compile your module using those intermediates. Once your module is cross-compiled, copy the .ko module to your running Raspberry Pi and you should be able to load it along with the stock kernel. The following is a quick how-to cross-compile the Raspberry Pi Kernel modules on Ubuntu 12.04:

First, know what version of RPi you're running:

uname -r and uname -a

root@raspberrypi:/# uname -r
3.2.27+
root@raspberrypi:/# uname -a
Linux raspberrypi 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l GNU/Linux

I've downloaded everything into a directory called ~/rpi on my Ubuntu machine.

1. Compile the Kernel

For 3.2.27+:

(instructions generally flow from: http://elinux.org/RPi_Kernel_Compilation)

  • Install Ubuntu
  • Get the rpi-3.2.27.tar.gz tarball
  • Get the tools tarball
  • tar xzf rpi-3.2.27.tar.gz and tar xzf master.tar.gz (tools)
  • Set the environment variable CCPREFIX:
    export CCPREFIX=
    /home/cowboy/rpi/tools-master/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-
  • Set the environment variable KERNEL_SRC:
    export KERNEL_SRC=/home/cowboy/rpi/linux-rpi-3.2.27
  • In KERNEL_SRC: execute "make mrproper" to ensure you have a clean kernel source tree.
  • Pull /proc/config.gz from your running Raspberry Pi and extract it into KERNEL_SRC on your Ubuntu machine.

    scp pi@rpiaddress:/proc/config.gz ./ (transfers it to Ubuntu)
    zcat config.gz > .config (uncompresses .config)
    mv .config $KERNEL_SRC (moves the uncompressed .config to KERNEL_SRC) 

  • On Ubuntu: Prime kernel with the old config:
    make ARCH=arm CROSS_COMPILE=${CCPREFIX} oldconfig
  • On Ubuntu: Build the kernel:
    make ARCH=arm CROSS_COMPILE=${CCPREFIX}

This will produce a fresh Module.symvers and since you're using the cross-compile tools, the intermediates the tools generate should be binaurally compatible with Raspberry Pi.

2. Compile your kernel module

Next, make a separate directory outside of KERNEL_SRC (or example: ~/pwm) and write a quick Makefile for your kernel module:

obj-m += rpi-pwm.o

all:
    make ARCH=arm CROSS_COMPILE=${CCPREFIX} -C /home/cowboy/rpi/linux-rpi-3.2.27 M=$(PWD) modules

clean:
    make -C /home/cowboy/rpi/linux-rpi-3.2.27 M=$(PWD) clean

(learn more about compiling Linux kernel modules here)

When you're ready to compile your module, go ahead and execute "make" and you should be gold. Copy your rpi-pwm.ko to your RPI then use modinfo rpi-pwm.ko to review the module information:

root@raspberrypi:/# modinfo rpi-pwm.ko
filename:       /home/pi/temp/rpi-pwm.ko
alias:          platform:bcm2708_pwm
author:         Sean Cross <xobs@xoblo.gs> for Adafruit Industries <www.adafruit.com>
license:        GPL
srcversion:     44D62553C321E91B0CB1996
depends:       
vermagic:       3.2.27 preempt mod_unload modversions ARMv6

Last but not least, you should be able to insmod rpi-pwm.ko without any problems. My dmesg:

root@raspberrypi:/# dmesg | tail -5
[18852.504621] rpi_pwm: disagrees about version of symbol module_layout
[19043.461377] rpi_pwm: disagrees about version of symbol module_layout
[21401.160720] rpi_pwm: disagrees about version of symbol module_layout
[22026.927225] rpi_pwm: disagrees about version of symbol module_layout
[26733.475758] Adafruit Industries' Raspberry Pi PWM driver v1.0

If you want your module to auto start:

  • Copy your .ko module to /lib/modules/3.2.27+
  • Edit /etc/modules and append your module to the end of the list.
    Mine looks like:

    # /etc/modules: kernel modules to load at boot time.
    #
    # This file contains the names of kernel modules that should be loaded
    # at boot time, one per line. Lines beginning with "#" are ignored.
    # Parameters can be specified after the module name.

    snd-bcm2835
    spi-bcm2708
    i2c-bcm2708
    i2c-dev
    rpi-pwm

  • Next, run: depmod -a
  • Next, run: modprobe <yourmodulename>

Then reboot and your module should auto start on boot.

Hope that helps!
Brian Chavez