Long Term Storage

I have this fantastic server which redirects console output to its serial port and I wanted to take advantage of it.  I went through the process of setting up inittab and grub to work and then I started testing the configuration.  In Windows using Putty, everything worked fine.  As soon as I moved the cable over to the serial port of an OSSIM box, I kept getting this error.

/dev/ttyS0 is not a tty

Googl’ing around returned nothing helpful which is why I am posting this here.

I have a habit of  disabling unused peripherals when I setup a new box.  Since I rarely use Serial Ports, I disable them.  Well.. that was the problem.  The disabled serial port isn’t going to work very well for a serial connection.

Hope this saves someone some time.

Lurking in the depths of the internet is a problem. A problem so large that it is going to cost billions to fix. It will effect every device connected to the internet without exception. Every cell phone, every game console, every computer, every router/modem, EVERYTHING. And it is a secret. Well, not really a secret, just not something anyone talks about.

The problem is the language that the internet speaks, is running out of unique names. Specifically, the IPv4 address space is running out of unassigned addresses. The simply version of why this is a problem is no new websites will be able to be online. It is a lot more complicated than that and will even impact users to a degree, but that is for a different article.

This is where the 600 days comes in. By the estimates of the people who are able to do estimates, the currently unused addresses will run out in about 600 days as of the beginning of 2010. As that day approaches, you can expect all sorts of shenanigans regarding pricing and allocation decisions. It will become much much more difficult and costly to setup your own website/service.

The good news is, there is a fix. The bad news is what I was saying in the beginning. It is going to be expensive as hell and it is going to impact a couple billion devices. The worst part is, you can’t even take steps to fix this yourself right now.

The answer is a new language. Internet Protocol Version 6 or IPv6. It solves the addressing problem for a VERY long time. The current version, IPv4, supports about 4.3 Billion addresses. This is represented by 32bits or 2^32nd. IPv6 supports 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses which is 128 bits or 2^128th. This post does a good job of expanding on what this means.

Because this is a new language, this means all of the devices have to be taught to speak it.  In the vast majority of cases, this is software and could be done for free.  The problem is if you are the manufacturer of such a device, why would you provide a free upgrade, when you could sell a new device?  This is further complicated, by the fact that essentially no one supports IPv6.  The deep insides of the internet do, but the majority of the pieces that are exposed on the internet, do not.  The biggest hurdle is most ISPs (Comcast, ATT, Verizon, Charter, Cable Vision, etc) don’t support IPv6 for their users.  Even if you could go buy replacement devices or upgrades to your equipment to support IPv6, you still don’t have access to the IPv6 Internet.

During the period of overlap when not everything speaks IPv6, we will run into problems of sites only being accessible from v4 or v6.  As time progresses that will go from overwhelmingly v4 to primarily v6 and this transition will take a very long time.  The general masses are going to learn more about networking than they wanted to know out of necessity.  Where did I put that number to tech support?

TL;DR; We have 600 days to make the Internet6 accessible.  After that, things start becoming REAL complicated, real quick.

Note:  This article is meant to build awareness not be complete or thurough.  There are large gloss-overs, simplifications and omissions to keep this from being a book.

As the subject says, this webserver is now serving the v6 Internet. At home, I am working on my v6 network with a 6in4 tunnel to Hurricane Electric. I have a Vyatta router which is my tunnel end point and a /48 for everything inside.

I am hoping to put together a post about my journey, but I suspect it will end up like the post I have been working on for my Vyatta router build.

Tiger warned me that IP fowarding was enabled this morning.  It’s behavior is definitely odd in terms of when it reports something is afoot.

NEW: --WARN-- [lin015w] The system has IP forwarding enabled


I digress.  Good article here on how to dis/enable IP Forwarding, but more importantly for my memory it also has the sysctl syntax which I forget on a regular basis.

Check if IP Forwarding is enabled

We have to query the sysctl kernel value net.ipv4.ip_forward to see if forwarding is enabled or not:
Using sysctl:

sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0

or just checking out the value in the /proc system:


cat /proc/sys/net/ipv4/ip_forward
0

As we can see in both the above examples this was disabled (as show by the value 0).
Enable IP Forwarding on the fly
As with any sysctl kernel parameters we can change the value of net.ipv4.ip_forward on the fly (without rebooting the system):

sysctl -w net.ipv4.ip_forward=1

or

echo 1 > /proc/sys/net/ipv4/ip_forward

the setting is changed instantly; the result will not be preserved after rebooting the system.
Permanent setting using /etc/sysctl.conf
If we want to make this configuration permanent the best way to do it is using the file /etc/sysctl.conf where we can add a line containing net.ipv4.ip_forward = 1

/etc/sysctl.conf:
net.ipv4.ip_forward = 1

if you already have an entry net.ipv4.ip_forward with the value 0 you can change that 1.
To enable the changes made in sysctl.conf you will need to run the command:

sysctl -p /etc/sysctl.conf


There is some good stuff down in the comments too.
Thanks MDLog.

Apparently Dell’s TPM VT Trusted Execution is incompatible with 64bit virtual guests.  You must go into the BIOS and disable the VT Trusted Execution in the virtualization settings to get it to work.  Alternative is to disable TPM since it is highly unlikely you are using it anyway. (DISCLAIMER:  Be sure your corporate WDE does rely on it.)  If you go this route, you must actually POWER OFF for the changes to take effect.

To fulfil the purpose of this site, I am reposting a collection of tools.  The Top 10 Security Assessment Tools.

In my efforts to build a new router, more to come on that topic, I had a need to make a ISO image bootable on a USB Drive.  My Googling let me to this simple, straight forward article.

Thanks Lnx2.

I have made some slight modifications and formatting to the original steps based on what I found to work.

There are two methods.  1 is to not use a partition table and write the file system directly to the device.  Method two is more traditional.

1. Method (/dev/sdX is your USB flash drive) :

A) Create a filesystem n the whole device without a partition table.(saves some space, and you don’t have to worry about the MBR)

sudo mkdosfs -I -v -n Ubuntu -F 32 /dev/sdX

B) Create “volume boot record” and the file ldlinux.sys using this command:

sudo syslinux /dev/sdX”

C) Mount the USB drive and the iso image file.

sudo mount -o loop /path/to/iso /mount/point
sudo mount /dev/sdX /mount/point

D) Copy all the files in the iso to the USB drive:

sudo cp -P --preserve=all -R /path/to/iso/* /path/to/iso/.* /usb/mount/point/

E) Rename the isolinux directory to syslinux and the isolinux/isolinux.cfg to syslinux/syslinux.cfg

sudo mv isolinux syslinux
sudo mv syslinux/isolinux.cfg syslinux/syslinux.cfg

F) reboot

2. Method (/dev/sdX is your USB flash drive):

A) Create a partition:

sudo parted /dev/sdX

In parted

mkpart primary 0% 100%
quit

B) Create a filesystem on the firstpartition

sudo mkdosfs -v -n Ubuntu -F 32 /dev/sdX1

C) Overwrite the MBR.

sudo dd if=/usr/lib/syslinux/mbr.bin of=/dev/sdX count=1

D) Create “volume boot record” and the file ldlinux.sys using this command:

sudo syslinux /dev/sdX1

E) mount the USB drive and the iso image file.

sudo mount -o loop /path/to/iso /mount/point
sudo mount /dev/sdX /mount/point

F)copy all the files in the iso to the USB drive:

sudo cp -P --preserve=all -R /path/to/iso/* /path/to/iso/.* /usb/mount/point/

F) rename the isolinux directory to syslinux and the isolinux/isolinux.cfg to syslinux/syslinux.cfg

sudo mv isolinux syslinux
sudo mv syslinux/isolinux.cfg syslinux/syslinux.cfg

G) reboot

As a follow up to last weeks post on NTFS-3G performance issues, I stumbled across this VMWare KB article today with some further tweeks that can reduce I/O operations by utilizing more system memory.

Same as last time.  Power-off the VM, close it in VMWare application, and edit the .vmx file.

Add the following:

MemTrimRate = "0"
mainMem.useNamedFile=false
sched.mem.pshare.enable = "FALSE"
prefvmx.useRecommendedLockedMemSize = "TRUE"

I haven’t tested it much so far, but I am willing to give anything a try to improve the performance of the VM.

Another VMWare issue I started having is Unity mode wouldn’t start.  Click the button, the unity menu pops up and the VMWare window minimizes, but the unity app never displays.

This is also  a common problem with a very simple fix.  Apparently, sometime in the past, I added a line to my /etc/vmware/config file to problem fix or enable some other feature.  This line and Unity just don’t mix.

The fix, per this thread and ultimately this blog post, is to remove the following line from your /etc/vmware/config file.

xkeymap.nokeycodeMap = true

Keep reading the VMWare Communities post for additional details if you added that line to address some key mapping issues.

Like many Linux users of VMWare, I live in a mixed world where not everything I need or want can be done in Linux so I have to resort to dual boots and virtual machines.  Combine that with the sad state of EXT2/3 FS drivers for Windows and you have a situation where you have make concessions just to get by.  For this particular story I am referring to having NTFS formated external hard drives.  I like to run my virtual machines on separate physical disks especially when the physical machine is an under-powered laptop.  So I have my VM on an NTFS external drive, trying to run it form my Linux desktop and I am getting terrible performance.  TERRIBLE.  Like the window graying out cause it is unresponsive and having to kill -9 processes just to get anything moving again.  Looking at the processes the FUSE ntfs-3g driver is pegged at 100% even after killing the vm.

It turns out that this is a known problem in the ntfs-3g driver and is being ‘worked on’.  I say ‘worked on’ cause the problem has apparently been around for a long time.  So long in fact, that it has made it to the FAQ on the ntfs-3g.org website here.  With all of that said, the solution is to simply add a line to your .vmx file.  Note:  This is a per-VM setting, so you will have to do it on all of them.

Edit your .vmx file and add the following.

mainMem.useNamedFile=FALSE

Do note as the state on the website that your /tmp dir must be on a non-ntfs-3g partition or else you could cause additional performance issues.

The following steps outline the process of using Public Key authentication for SSH with Putty as your client.  If you follow the entire guide and get it right, you will not have to type in your username/password for the servers you have setup.

Install Files
Copy putty.exe, puttygen.exe, pscp.exe, pftp.exe and pageant.exe to c:\windows\system32

Generat your key pair.

Start -> Run -> puttygen
Change the “Number of bits in a generated key” from 1024 to 2048 or what ever value you would like.
Click Generate
You will need to move the mouse around in the area with the progress bar till the bar completes.  This process is generating random data to create your key from.
Once complete, it will display the key contents.
In the key-comment box, change the value to something you will recognize.
Enter a passphrase and repeat it.
Click the “Save private key” button and save your private key.  I saved mine to My Documents with the same name I placed in the comments.
Click the “Save public key” button and do the same.  I changed the extension to .pub for identification.
Next you need to copy the Public key from the top box.  Be sure to get the entire thing.  It will start with ssh-rsa and end with what you typed in the comments.
Paste the contents below replacing the line that says “<Paste authorized key data here, overwriting this line>”.  None of the existing text should remain.
Log onto each server and paste the following lines.  This will create the .ssh folder, authorized_keys file, and correct the permissions.

cd
mkdir .ssh
chmod 0700 .ssh
cat << EOF > .ssh/authorized_keys
<Paste authorized key data here, overwriting this line>
EOF
chmod 0600 .ssh/authorized_keys


Close puttygen.

Setup your saved sessions in Putty.

Start -> Run -> putty

Adjust Default Session.
These steps will set the default settins of your instance of Putty to use the private key you just generated.

In the Saved Sessions box, select Default Settings and click load.
Browse the tree on the left to Connection -> SSH ->Data
Enter your username in the Auto-login username box.
Browse the tree on the left to Connection -> SSH -> Auth
Click the browse button and select the private key you saved earlier.  It will have a .ppk extension.
Browse back to the Session section on the left.
Click Save.

Existing Sessions.
Repeat this process for each server you created the authorized_keys file on either by changing an existing saved session or creating a new one.

Testing Public Key authentication
Using a session you have setup as above, connect to the server.
You should be prompted for your key passphrase.  If you are not, you have done something wrong and you need to troubleshoot.

Once you get it working, move onto the Putty Agent section.

Putty Agent
The following steps create a shortcut in the startup folder so each time you logon the Putty Agent will automatically start and load the private key.

Start -> All Programs -> Right-click on Startup, click open.
Right-click on the window, select New -> Shortcut
In the field enter: pageant "c:\path\to\private_key.ppk"
Mind the quotes around the path.
Click next.
Give it a name such as “Putty Agent”
Click finish.

Test Public Key authentication with Putty Agent.

Start -> All Programs -> Startup -> Putty Agent (Or what ever you named the shortcut.)
You will be promoted to enter the passphrase for your privage key.
Enter the passphrase.
You will now see a little computer screen with a hat on it in the system tray.
Open putty and use one the sessions setup previously for Public Key authentication.
This should log you in automatically with out prompting for a password or passphrase.

SAIC announced today that they are starting a CyberSecurity blog over at Typepad.  It should be interesting to see who they have do write ups.

As you use Ubuntu, you are bound to come across a time when you want to build a custom file association.  Inevitably, you are bound to make a mistake in that file association and end up with a second entry in your available applications that is wrong.

I managed to do this while installing uTorrent via CrossOver.  The entry in the available application field also had a funky ^C5 something something something for the “mirco” which translated itself as the entry name automagically.

When I managed to get it all sorted out, I ended up with two entries in there.  One working with a crap-tastic name and the other not working with the right now.  That is when I started digging into how to manually create/remove those entires.

It turns out that it is very easy.

The available associations are located in ~/.local/share/applications

It should be fairly obvious which one you need to edit/delete when you list the contents of that directory.

This is the resulting file I had after correcting and assigning the icon.

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Icon=/home/jon/.cxoffice/utorrent/windata/Desktop.C^5E3A^5Fwindows^5Fprofiles^5Fcrossover^5FDesktop/^C2^B5Torrent.xpm
Exec="/home/jon/.cxoffice/utorrent/desktopdata/cxmenu/StartMenu.C^5E3A^5Fwindows^5Fprofiles^5Fcrossover^5FStart^2BMenu/^C2^B5Torrent" %f
Name=uTorrent
Comment=Custom definition for crossover uTorrent
NoDisplay=true


If you use this, a couple of things to note. My username is jon and the bottle I created in CrossOver is called utorrent. If you adjust it for those two changes, you should be able to copy/paste and have a file association ready to go for uTorrent including a spiffy icon.

Anyone who runs GNU/Linux is bound to need a console at some point.  Sometimes it is just easier to switch over to a virtual console and do what you need to do.  The default state of most consoles is an 80×30 character display which is basically useless if you want to see any kind of information.

The simple solution to this minimalist display is to set the vga kernel flag.  This allows you to set the resolution on your console framebuffer to something more useful.

The default settings example shows vga=791.  This puts your console in a 1024×768 resultion.  Which while not great is a significate boost over the standard 800×600.  If you simply enable this resolution, you will notice the Ubuntu usplash logo off center.  Very annoying.  The following steps are needed to make your Console Framebuffer something to look at.

1. Figure out what resolution your primary monitor supports.  This is typically what you have your desktop resolution set to.

2. Determine what your vga kernel flag needs to be set to.

The Wikipedia article on VESA BIOS Extensions will probably answer your questions.  Specifically, the section on Linux Video Modes.

For us widescreen users, we are not as lucky with our resolution choices.  Though, through my expirimentation, I did find that vga=840 works to give me a 1400×1050@16bit on my Dell Inspiron 1505 with the NVidia 7300 Go.  This is actually VESA 348 and does not follow the 512 rule stated in the Wiki article.  My assumption about why is probably due to manufacturers implementation.

Note: Don’t worry too much about getting it wrong.  One of two things will happen.  1. You will get no display or 2. you will get a grub screen telling you your video mode is unsupported.  I will cover how to correct this in a moment.

3. Edit your menu.lst file and add the vga= option in two places.  First, add it to the default options line.  You are looking for this line.
# defoptions=quiet splash

Change it to:
# defoptions=quiet splash vga=840

Be sure to set the 840 to the mode that represents your preferences.

Next change the default Boot menu kernel entry at the bottom. In a standard Ubuntu build, the default entry will be the first entry following this line.

## ## End Default Options ##

It will look something like this:
title Ubuntu 8.10, kernel 2.6.27-9-generic
uuid 636dc411-e53a-4776-a9e9-4fc9e277f445
kernel /boot/vmlinuz-2.6.27-9-generic root=UUID=636dc411-e53a-4776-a9e9-4fc9e277f445 ro quiet splash
initrd /boot/initrd.img-2.6.27-9-generic
quiet

You need to add the vga=### to the end of the kernel line so it looks like this.

kernel /boot/vmlinuz-2.6.27-9-generic root=UUID=636dc411-e53a-4776-a9e9-4fc9e277f445 ro quiet splash vga=840

4. Next you need to update the usplash config to match the resolution.  This is how you keep the logo centered.

sudo vi /etc/usplash.conf

Change the x and y resolution lines to match your chosen resolution.

# Usplash configuration file
xres=1400
yres=1050

5.  Update your initramfs to take advantage of the usplash settings change.

sudo update-initramfs -u

6. Reboot.

You should see a smaller, higher resolution Ubuntu logo and then Gnome startup.

Upon reboot, if you get a blank screen, that means you chose a resolution that is larger than what your display can support. In this case, you will need to boot your Ubuntu CD and chose rescue mode. From there, chose a command prompt for the rescue CD. CD to /target/boot/gurb. then edit menu.lst with nano. Remove the vga=840 line from the kernel entry at the bottom.

If you are getting a grub menu saying you have set and invalid mode, press the space bar to see a list of valid modes. If you would like to get a complete list. Type in scan and press enter. This table is what I used to help determine the 840 setting for my display.  Select the letter representing your choice and your machine will continue to boot.

The entries listed on the table are the only VESA modes your video card support. Find the entry that best matches your display preferences without exceeding the max resolution of your monitor.  Take that number, add 512 to it and update the menu.lst file.  If this results in the invalid mode error again, you will need to experiment to find the actual setting.

Note:  The number following the resolution is the color depth or the number of bits being used to describe color.  Unless you have a specific limitation, it is safe and preferable to chose the highest number following your chosen resolution.  You will typically see 8,16, and 32.

My monitor supports 1650×1050 max resolution. The max resolution my video card supports is 1600×1200. Since the 1200 is greater than the 1050, if I chose this resolution, my monitor will not display and/or give me an error indicating that it is out of range. In my case, my next best choice was 1400×1050@16bit. The menu displayed this resolution as VESA 348. My next step was to convert the VESA mode to a linux VGA mode. Per the VESA BIOS Wiki I linker earlier, the standard is to add 512 to the VESA mode which would give me 860. I set vga=860 and rebooted. Same problem but it gave me an error stating 361 was not a valid mode. Since my goal was 348, I tried decreasing it by 12. I set vga=853 and rebooted. I got the grub error again, but it said that 355 was not a valid mode. Some simple math showed that I moved from 361 to 355 by subtracting 12 from the VGA mode, this time I need to move 7. I subtracted 13 this time which gave me the 840 and no grub errors on boot.

I have a Ubuntu 8.10 AMD64 Server.  In the syslog, I was getting the following error every few minutes.

console-kit-daemon[5013]: CRITICAL: cannot initialize libpolkit

The error is triggered by the update-modt cron job which runs ever 10 minutes.

This is a bug in Intrepid.  console-kit-daemon requires PolicyKit as a dependancy, but Intrepid (Server AMD64) does not install it when it installs console-daemon-kit.

The simple fix is to install policykit.

sudo apt-get install policykit

Next run of the update-motd job and the error is gone.