Sunday, December 29, 2013

Useful commands and scripts (Linux, Windows, bash, etc...)

Linux

Automatically call 'ls' when the directory is changed:
sudo echo export 'PROMPT_COMMAND='\''[[ $my_currdir != $PWD ]] && ls; my_currdir=$PWD'\''' > ~/.bash_aliases


Reinstall all installed packages with a specific word: (in this case, this line reinstalls all installed packages with the word perl in it)
sudo apt-get --reinstall install $(dpkg --get-selections | grep perl | awk '{print $1}')

(this case, this line reinstalls all installed packages with the word perl in it and not perl-base)
sudo apt-get --reinstall install $(dpkg --get-selections | grep perl | grep -v perl-base | awk '{print $1}')

Update all opkg packages
for pkg in `opkg list-upgradable | cut -d' ' -f1 | grep -v Multiple`; do opkg upgrade $pkg; done


Find all files in home directory larger than a defined size
find ~ -size 20MB

Find occurences of ip address by count:
grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' NCL-G2-LOG-1.txt | sort -r | uniq -c | sort -rn | head


Encrypted netcat group chat:
server: 
while true; do read -n30 ui; echo $ui |openssl enc -aes-256-cbc -a -k PaSSw; done | nc -l -p 8877 | while read so; do decoded_so=`echo "$so"| openssl enc -d -a -aes-256-cbc -k PaSSw`; echo -e "Incoming: $decoded_so"; done
client: 
while true; do read -n30 ui; echo $ui |openssl enc -aes-256-cbc -a -k PaSSw; done | nc $ipaddress 8877 | while read so; do decoded_so=`echo "$so"| openssl enc -d -a -aes-256-cbc -k PaSSw`; echo -e "Incoming: $decoded_so"; done

Centos issues:
when attempting to do an ssh-keygen as a user that is not in the sudoers file you will get a "permission denied" error because SELinux is blocking ssh-keygen. To get SELinux to accept the command without blockage, run the following:
chcon -R -t ssh_home_t ~/.ssh


Docker pull all and update all images from repositories:
docker images | awk 'NR > 1 && $1 !~ /<none>/ {print $1}' | uniq | xargs -n 1 docker pull


Mac

dd with a progress bar (requires brew and install pv):
pv -tpreb ~/Desktop/openSuse.img | sudo dd of=/dev/disk2 bs=1m


mount smb share:

mount_smbfs
or
mount -t smbfs

Windows

Flush Windows Updates cache:
net stop wuauserv
cd /d %windir%
rd /s SoftwareDistribution
net start wuauserv


Bash scripts

Auto extract compressed file (place the following in your bash_profile or bashrc file):
extract() { 
    if [ -f $1 ] ; then 
      case $1 in 
        *.tar.bz2)   tar xjf $1     ;; 
        *.tar.gz)    tar xzf $1     ;; 
        *.bz2)       bunzip2 $1     ;; 
        *.rar)       unrar e $1     ;; 
        *.gz)        gunzip $1      ;; 
        *.tar)       tar xf $1      ;; 
        *.tbz2)      tar xjf $1     ;; 
        *.tgz)       tar xzf $1     ;; 
        *.zip)       unzip $1       ;; 
        *.Z)         uncompress $1  ;; 
        *.7z)        7z x $1        ;; 
        *)     echo "'$1' cannot be extracted via extract()" ;; 
         esac 
     else 
         echo "'$1' is not a valid file" 
     fi 
}

Thursday, December 5, 2013

Windows XP Mode Crash After Domain Controller Migration

It appears that windows xp virtual pc retains some settings that cannot be removed after installation of the virtual machine itself.  When the integration features are turned on in the settings of the virtual pc (mainly network drives), the virtual machine tries to link it's previous route to the old domain controller with the new network drives which will cause the virtual pc to crash.  If you turn integration features off, the virtual will run odd with strange graphic artifacts and other nuances that shouldn't be there, but the point is that it will run.  To fix this, disable the network drives and keep the integration features turned on.  A "gpupdate /force" on the virtual pc may fix this drive issue but I have not tested it.

Tuesday, December 3, 2013

Headless Raspberry Pi media server

I had installed a raspberry pi in my old car and just played around with it.  Didn't really use it for anything spectacular except to say that i have a usb powered computer in my car.

Now I am exploring the possibility of making the pi a media server.  I have read a lot about media servers and have yet to find a decent method for multi-source music.  I am mainly talking about wifi, usb, bluetooth audio.  Mainly bluetooth streaming with MPRIS2 support for wireless phone control, stick it in a glove box or center console and forget about it.  Strange how nearly every car radio you can buy either has, or makes available to option as, a bluetooth receiver for phone calls and bluetooth streaming.  I have yet to find a linux application that combines these two elements together, as well as utilizing it as a media player.  In a nutshell, I would like to implement the ford sync system in the raspberry pi.  It doesn't have to look pretty or function the same, but it's just shocking how car audio systems are the best at being media players.

Current media players:
XBMC
Amarok
Clementine
Banshee
Volumio(Raspyfi(raspberry pi distro) replacement)
Pimusicbox (another raspberry pi distro that is actually really good)

Media/music servers/mixers:
XBMC
MPD
ALSA
pulseaudio
Mopidy (preferred)

Proposed implementation:
XBMC with a Mopidy backend for music playback with support for bluetooth, wifi, and usb audio reception.


http://www.ccoffey.ie/?p=53

Sunday, November 17, 2013

TP-Link MR3020 ExtRoot and other tips

The following is my working representation of an extroot implementation on a bleeding edge openwrt version r38886 on a mr3020 router with a 16gb flash drive running as root on a usb hub.

I have decided to use the extroot method because the bleeding edge image only leaves about a half of a megabyte of space left for packages.  And once you run out of space, there's almost no recovery option at this point because you're in a catch 22.  If you try to remove packages, it required temporary space in the overlay directory, which it doesn't have because of prior package installations used up all of that extra space.
So reimage your router and immediately do the following before installing any more packages, storage is minimal on this device.
You have 2 choices for extroot, an Overlay boot or a Root boot.  Overlay holds all of the packages kind of like an external hard drive for your router.  Root holds your entire root directory including all settings associated with the filesystem.  This is kind of unique because you could potentially have 2 completely different OS setups for your router without modifying anything, simple plug-n-play os.
For example, I use my router in my car with 2 different wifi usb adapters attached preconfigured to two different modes of adapter settings.  This means the /etc/config/network and the /etc/config/wireless files are tailored for the setup, which are a pain to begin with.  If i use the router on the go with tethering and tftp booting, reconfiguring these settings are an even bigger pain.  So you could essentially boot two different flash drives with the router configured for the two different environments with no additional configuration, as long your router is configured to boot to root.

Following these exact instructions in here http://wiki.openwrt.org/doc/howto/extroot#openwrt.12.09.attitude.adjustment

or follow my instructions below...

format a usb flash drive with 2 partitions of your choice, you can do this on the device itself but I wanted to make sure that the partitions formatted correctly so I used my ubuntu linux distro with gparted just to be safe since I know that works.
I have a 16gb sandisk flash drive model number SDCZ33-0160G-B35 partitioned to 2gb of swap as sda2 and the remainder storage of ext4 as sda1.  Be aware of where your partitions are (example: ext4 as sda1 and swap as sda2), the partition locations matter!!!

on the openwrt device ssh and install the following packages required for usb communication and storage:

opkg install block-mount kmod-usb-storage kmod-fs-ext4

then reboot
and run the following:
(the follwing lines mount the ext4 partition to a directory.  Make a temporary directory for the root or overlay filesystem.  By binding the directory you are not interfering with active memory currently running on your system.  Then on line 3, a complete duplicate of the root filesystem is made and copied to the ext4 partition of the usb flash drive which is mounted on /mnt/sda1.   Then unmounted.)
(Also, if anyone has used chroot before, the following process should look familiar)

#this is not a runnable line!: Change "mount --bind / /tmp/cproot" to "mount --bind /Overlay /tmp/cproot" for overlay copy instead of the root filesystem.#

mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1
mkdir -p /tmp/cproot
mount --bind / /tmp/cproot
tar -C /tmp/cproot -cvf - . | tar -C /mnt/sda1 -xf -
umount /tmp/cproot

Make sure your fstab looks EXACTLY as below.  Depending on your MR3020 configuration and usb flash drive, it may not settle in time for the fstab to initiate it as a root file system.  Therefore the line in this config file that needs the most attention is the one containing delay_root.  This is for an overlay extension.  If you want a root extension (also called pivot root), change the option target to '/' in config mount.

__________________________

vim /etc/config/fstab

config global

option anon_swap '1'
option anon_mount '1'
option auto_swap '1'
option auto_mount '1'
option delay_root '10'
option check_fs '0'

config mount

option device '/dev/sda1'
option target '/'
option fstype 'ext4'
option options 'rw,sync'
option enabled '1'
option enabled_fsck '0'


config swap

option device '/dev/sda2'
option enabled '1'

_____________________________

Reboot, and it should be all booted up from the flash drive like magic.  But you won't know the difference unless if you run df -h.

Luci is NOT going to work after performing a snapshot so do the following to get your web interface back up and running.  *** due to a bug as of 06/17/2014, luci-ssl is required for uhttpd to start properly
opkg update
opkg install luci
opkg install luci-ssl
/etc/init.d/uhttpd enable
/etc/init.d/uhttpd start

You may also encounter an issue where luci would not be able to find the theme that gets preinstalled with luci.  This is due to a misconfiguration with the bootstrap theme package where it does not update the luci config file in /etc/config/.  To fix, simply add the following to the end of the luci file:

config internal themes 
        option  OpenWrt '/luci-static/openwrt.org'  
        option  Bootstrap '/luci-static/bootstrap'

Since luci doesn't provide a great way for upgrading packages, here is the alternative:
opkg update
for pkg in `opkg list-upgradable | cut -d' ' -f1 | grep -v Multiple`; do opkg upgrade $pkg; done

if there is an instance where you need to get back to stock, run the mr3020 failsafe or if you have internet access and ssh/telnet access to the router (like I have needed on multiple occasions), run this (assuming you're using squashfs):

"firstboot" (this is like a fresh install of OpenWrt)

or

cd /tmp
wget http://downloads.openwrt.org/attitude_adjustment/12.09/ar71xx/generic/openwrt-ar71xx-generic-tl-mr3020-v1-squashfs-factory.bin
mtd -r write openwrt-ar71xx-generic-tl-mr3020-v1-squashfs-factory.bin firmware
(this completely reflashes the router using the supplied bin file)

Headless Raspbian Installation

I had an issue trying to ssh into a headless raspberry pi installation using the latest raspbian image as of writing this on 11/17/2013.  After a bit of research the easiest way is to assign a default ip address to the cmdline.txt file located in the first partition of the sd card.  Add ip=169.243.0.2 (or whatever ip address you would like) to the end of the second line of the file.  Save and reboot.  Assign your comp a static ip address within the ip range of your ip address.  I assigned mine statically as 169.243.0.1.  You should have no problem sshing now and run rpi-config to finish the installation.

After you ssh over ethernet and you want to setup your WPA2 wifi connection with a connected usb wifi adapter, edit the file /etc/wpa_supplicant/wpa_supplicant.conf and add the following:

network={
        ssid="wirelessnetwork"
        psk="password"
        proto=RSN
        key_mgmt=WPA-PSK
        pairwise=CCMP
        auth_alg=OPEN
}


If you are like me and use X instead of vnc or rdp, there is a slight bug in Raspbian as of March 8, 2014 that will render your lxsession through your X-client (XQuartz for mac) with a non-responsive wallpaper and no X throughput.
To fix this:
1. Run lxsession once to create config files.
2. In an ssh session run : cp -rp /etc/xdg/lxpanel/profile/LXDE/* ~/.config/lxpanel/LXDE/
(found here http://www.raspberrypi.org/phpBB3/viewtopic.php?p=245144#p245144)
3. Run lxsession again and you should have a working lxsession ready to go

Thursday, November 14, 2013

PLX Devices Project

For my college project, I am pursuing an open-source solution to extracting and monitoring raw data coming from any OBD-II enabled vehicle.  Most likely this will be web-based and will communicate through straight tcp/ip over wifi.

Hopefully this will become a direct replacement for dash command, but less pretty and more functional.

I will update with a link to the source code as I progress with this project.

Update: 11/15/2013
I have successfully received the rpm of my jeep from the kiwi 2 wifi module using netcat.  Simply by connecting over wifi using a laptop (mac in my case) and issuing a couple of commands, my rpm has populated in straight ascii over tcp/ip.  When sending a message to request the rpm for example, we ask the car to shoot back the result from mode 1 pid 0c as messaged 010c.  To delimit the request we have to add a "\r"... but not in the literal sense.  This is a carriage return and submitting an ascii value of this as 0d is dependent upon the operating system and keyboard layout.  I don't know what it is on windows but on mac you have to type a Control+v enter enter.  yes, two enters.  When using a packet sniffer you see the payload as 303130630d0a.  The car responds and shoots back your response.

To do:
when accessing the car through netcat, the connection times out rather quickly if i don't send any response within 10 seconds and it terminated by the kiwi 2 wifi device.  I have to find a way to prolong the connection.
Develop an application like dashcommand except open source.  Most likely going to be web-based.

Update 11/23/2013
I have successfully connected to the car using a project called WebTCP and it works!
I requested the available PIDs by sending 0100\r\n and it returned BF BE B9 90 which corresponds to 10111111101111101011100110010000.  These values are the PID numbers that are available discerned by either a 1 or 0 as true or false.  For example, 1011 means PID 1,3,4 are available.  This is all in mode 1.  Next step, using squel.js to query the appropriate values.

3+ booting on a macbook pro 2009 5,5

My 13" macbook pro 5,5 is currently booting Mac OSX 10.9, Windows 7 32-bit, Ubuntu 13.10, Kali 1.0.5, OpenSuse 13.1, and BackTrack R2.  It currently hold two internal hard drives (took out the superdrive) which I have partitioned the first one with OSX and Windows and the second with my linux distros.

I am using this very handy tool called rEFInd... get it, efi?
Anyway, this, in combination with grub2, allows me to boot basically any quantity of operating systems I want to (hdd space limited).

A few things to mention concerning the limitations of mac booting.

Grub is needed as they boot through the legacy method as well as windows.  Apple decided to make their macbooks and all successor computers including the iMacs with this thing called a hybrid mbr.  This is a terrible and highly unreliable method for having the capability of having a bios-less boot using the mbr as it's primary boot method while still having the capability of efi booting, and even worse you can only have 4 boot entries in the hybrid mbr.  Mac,   The hybrid mgr was designed solely for the purpose of booting windows since efi is only supported in windows 8 and higher.
Also, typical bios-based usb booting will NOT work... at least on the macbook pro 5,5.  I have not tested bios based usb booting on other macs so results may vary.  So the solution is to EFI boot your usb distro.  Even though bios-based booting is the most common type, EFI booting is quickly taking over and will become the standard in booting any os in the future which makes your mac future-proof when bios-booting becomes outdated.

The following methods have worked for me when installing windows with bootcamp:

In your macintosh operating (whatever it may be), install bootcamp with your choice of windows os.  Make sure you have allocated enough space for the windows partition to allow for other linux distributions as they will installed in the windows space.  You can actually use bootcamp to install a linux distro, though I have never tried it.  Using bootcamp allows the hybrid mbr to function properly with the other linux distros who boot in typical mbr style. Once the installation is complete and you have installed windows, now download and install refind from http://www.rodsbooks.com/refind/getting.html
Install from mac using terminal.  Reboot twice (this is because of funky mac bios firmware failing to boot the first time, a hard reboot maybe required if it freezes).
Now it's linux time.  I like ubuntu and I know it works so we'll be installing this.  Make a bootable usb drive with ubuntu installed using these instructions http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx
The first installation is always the hardest because we use the installation as the booting point for the rest of the distros because of the hybrid mbr limitation.

More procedures to come.

Booting off of the internet

I have successfully booted ubuntu 10.10 off of the internet using netbootme.me.
This is a really neat and unfortunately a project that is no longer maintained.

I am using the tftp dhcp method.
In a nutshell, I have told my openwrt-based router to tftp boot the netbootme.kpxe from the specified directory and that in turn goes out to the internet and boots a very basic boot iso from another tftp server tftp.netbootme.me.
Then I am given a choice of other bootable iso's to boot from and for this test I chose ubuntu 10.10 installer which was completely successful.  The tftp server method is very unreliable, so it takes a couple tries to get the basic bootable iso from the internet after the kpxe is loaded.

This kpxe is based off of gpxe which is again, no longer maintained.  The successor is called iPXE which is the same thing as gpxe and then some.  I will be hosting my own tftp server with iPXE on a bootable USB drive so that I may boot my own iso's from the internet.

netboot.xyz
http://netbootxyz.readthedocs.org/en/latest/

Cross platform multi-monitor multi-os support

 I have a couple old laptops lying around and always would've like to convert that dusty hardware into something I can use everyday.
Use these laptops as external monitors through software, no hardware hacking.
Speed can be a perk but, I'm just looking for the ability first.  So far I have not found a reliable solution.  The technology is there and currently exists as evident below but, the initiative has not advanced.

Why I am not the only one that want's this and concludes the precise reasoning of why this should be implemented: http://www.raspberrypi.org/phpBB3/viewtopic.php?f=41&t=12100&p=453357#p453357

Collected information:
displaylink - usb monitor support (would be awesome if these plug-n-play monitors like the 16" AOC usb-powered one worked the same way as another laptop connected by usb with a display driver that emulates a monitor)

xdmx - successful implementation of Windows Linux OSX multimonitor multisystem (very buggy, not very reliable, uses ssh over network, uses cygwin as an emulator for an x terminal, uses only x server for display output, no longer maintained as of 2004)

maxivista/zonescreen - the only implementation that I know of the successfully displays multiple screens over tcp/ip

piwall - libav based projection of imagery over 9 different monitors over the network using a player capable of projecting an image over multiple monitors at the server end.

Mobile Device Display - iDisplay, AirDisplay, for ios

Open Source monitor emulation:
http://www.ashokraja.me/tips/How-to-enable-dual-or-multi-monitor-support-in-Oracle-Virtual-Box

Best case scenario - open-source zero-conf cross-platform monitor detection over any udp-capable medium (cat5, wifi, bluetooth, etc...).

Open source working monitor support:
http://www.spice-space.org/home.html

New best case scenario: avahi serviced monitor detection using spice for windows and X for linux distros.  Server can only be run on linux.  Needs functionality to use windows as the server or implement a p2p instead of a server/client relationship.  Events would be transported over spice but interpreted by synergy.

Intro

This blog contains my tech ideas/projects and information need to implement these on a later date or at least attempt to do so