Rice University logo
 
Top blue bar image Kiran Thyagaraja
Brain Dump
 

How to replace GPT-labeled drives in a software RAID

February 10th, 2016 by kkt3863

For the sake of this post, lets assume that you have a RAID 1 device setup on Linux.

First identify whether your RAID is healthy. If it is, this is what it looks like:

[root@node ~]# cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sda2[0] sdb2[1]
3650885632 blocks super 1.2 [2/2] [UU]
bitmap: 0/28 pages [0KB], 65536KB chunk

md127 : active raid1 sda1[0] sdb1[1]
255868928 blocks super 1.2 [2/2] [UU]
bitmap: 1/2 pages [4KB], 65536KB chunk

unused devices:

If the RAID device is NOT healthy, the output looks like this:

[root@node ~]# cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sda2[1]
3650885632 blocks super 1.2 [2/1] [_U]
bitmap: 4/28 pages [16KB], 65536KB chunk

md127 : active raid1 sda1[1]
255868928 blocks super 1.2 [2/1] [_U]
bitmap: 2/2 pages [8KB], 65536KB chunk

unused devices:

We can see that we only see block device sda active on both RAID devices. We can see that sdb has disappeared. Lets check whether sdb is indeed dead.

[root@node ~]# smartctl -i /dev/sdb
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-229.el7.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor: HP
Product: SOMEPRODUCTID
Revision: HPD5
Logical block provisioning type unreported, LBPME=-1, LBPRZ=0
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Logical Unit id: 0x5000c500846bacf7
Serial number: SOMESERIAL
Device type: disk
Transport protocol: SAS
Local Time is: Wed Feb 10 10:25:11 2016 CST
device is NOT READY (e.g. spun down, busy)
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

Lets proceed to removing this device from the RAID, if it has already not removed automatically.

[root@node ~]# mdadm --manage /dev/md126 --fail /dev/sdb2
[root@node ~]# mdadm --manage /dev/md126 --remove /dev/sdb2
[root@node ~]# mdadm --manage /dev/md127 --fail /dev/sdb1
[root@node ~]# mdadm --manage /dev/md127 --remove /dev/sdb1

Now, go to the server and physically remove the old, dead drive and replace it with a healthy, new one. You can physically identify the dead drive (and its serial number) from the output from smartctl -i output.

Once the server is booted up, set up the new drive. To do so, we need a package called gdisk. Install gdisk on the server and clone the GPT partition table.

[root@c125 ~]# sgdisk -R /dev/sdb /dev/sda
The operation has completed successfully.
[root@c125 ~]# sgdisk -G /dev/sdb
The operation has completed successfully.

The first command replicates the partition table from the good RAID drive into the newly added drive. The second command randomizes the GPT UIDs on the second drive, just so that the drive is not an exact clone.

Add the newly readied drive into the RAID.

[root@c125 ~]# mdadm --manage /dev/md126 --add /dev/sdb2
mdadm: added /dev/sdb2
[root@c125 ~]# mdadm --manage /dev/md127 --add /dev/sdb1
mdadm: added /dev/sdb1

Now wait until the drives are synced and release the server into the wild. Well, not literally.

Compiling GCC 4.3.2 and MPICH2 1.1 on Blue Gene/P

April 29th, 2014 by kkt3863

Compiling GCC 4.3.2 and MPICH2 1.1 for the Blue Gene/P

GCC 4.3.2

1. First install GMP and MPFR

GMP

wget ftp://ftp.gnu.org/gnu/gmp/gmp-6.0.0a.tar.bz2
tar xf gmp-6.0.0a.tar.bz2
cd gmp-6.0.0a
./configure CFLAGS="-O3" CC=powerpc-bgp-linux-gcc --prefix=/opt/apps/gcc/4.3.2/ --host=powerpc-bgp-linux --with-gmp=/opt/apps/gcc/4.3.2/

Note: While running “make”, several files are generated using newly compiled binaries. Since these binaries are cross-compiled, the build process fails, so you have to manually build this one binary and then generate the necessary file like so:

gcc -m32 -o gen-psqr gen-psqr.c
gen-psqr 32 0 >mpn/perfsqr.h
make install

MPFR

wget ftp://ftp.gnu.org/gnu/mpfr/mpfr-3.1.2.tar.bz2
tar xf mpfr-3.1.2.tar.bz2 
cd mpfr-3.1.2/
./configure CFLAGS="-O3" CC=powerpc-bgp-linux-gcc --prefix=/opt/apps/gcc/4.3.2/ --host=powerpc-bgp-linux --with-gmp=/opt/apps/gcc/4.3.2/
make install

* Add /opt/apps/gcc/4.3.2/lib into /etc/ld.so.conf.d/gcc.conf and run ldconfig. This is needed for the temp gcc to compile glibc in the toolchain.

cat /bgsys/drivers//ppcfloor/toolchain/README.toolchain.gomp 
cp /bgsys/drivers//ppcfloor/toolchain/buildGompToolChain.sh .
./buildGompToolChain.sh /bgsys/drivers/ppcfloor /home/kkt3863/src/gcc-4.3.2-toolchain /opt/apps/gcc/4.3.2 /opt/apps/gcc/4.3.2 /bgsys/linux/1.4.220110323/usr/src/linux/include

MPICH2 1.1

1. First fetch the dmcf source repo from ANL

git clone http://dcmf.anl-external.org/dcmf.git

2. cd dmcf/lib/mpich2

This directory contains the patches to mpich2 source directory

3. Fetch mpich2 1.1 (This version only) and untar

wget http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/downloads/tarballs/1.1/mpich2-1.1.tar.gz
tar xf mpich2-1.1.tar.gz

3. Apply the patches from DMCF into MPICH2

zcat ~/src/dcmf/lib/mpich2/patch.gz | patch -p0 --force --directory=~/src/mpich2-1.1
zcat ~/src/dcmf/lib/mpich2/patch.gz | ~/src/dcmf/tools/patch-delete --directory=~/src/mpich2-1.1

4. The autoconf tool on BGP is old. So tar up the newly patched mpich2 directory and then run maint/updatefiles on Davinci
(On BGP)

tar cf mpich2-1.1.tar mpich2-1.1

(On davinci)

tar xf mpich2-1.1.tar
cd mpich2-1.1
./maint/updatefiles

Repackage the mpich2 directory and scp it back to BGP

cd src/mpich2-1.1
touch --no-create Makefile.in

5. In the mpich2-1.1 source directory mkdir a build directory and cd into it. Use the following configure line to configure the source

GCC="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-gcc" \
CC="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-gcc" \
CXX="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-g++" \
F77="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-gfortran -fno-underscoring" \
F90="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-gfortran -fno-underscoring" \
AR="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-ar" \
LD="/opt/apps/gcc/4.3.2/gnu-linux/bin/powerpc-bgp-linux-ld" \
MPICH2LIB_CFLAGS="-g -Wall -O3 -mcpu=450fp2 -mstrict-align -fno-strict-aliasing" \
MPICH2LIB_CXXFLAGS="-g -Wall -O3 -mcpu=450fp2 -mstrict-align -fno-strict-aliasing" \
MPICH2LIB_FFLAGS="-g -Wall -O3 -mcpu=450fp2 -mstrict-align -fno-strict-aliasing" \
MPICH2LIB_F90FLAGS="-g -Wall -O3 -mcpu=450fp2 -mstrict-align -fno-strict-aliasing" \
CFLAGS="-I/bgsys/drivers/ppcfloor/comm/sys/include" \
CXXFLAGS="-I/bgsys/drivers/ppcfloor/comm/sys/include" \
FFLAGS="-I/bgsys/drivers/ppcfloor/comm/sys/include" \
F90FLAGS="-I/bgsys/drivers/ppcfloor/comm/sys/include" \
LDFLAGS="" \
MPILIBNAME="mpich.cnk" \
MPICXXLIBNAME="cxxmpich.cnk" \
MSGLAYER_INCLUDE="-I/bgsys/drivers/ppcfloor/comm/sys/include" \
MSGLAYER_LIB="-L/bgsys/drivers/ppcfloor/comm/sys/lib -Wl,-rpath,/bgsys/drivers/ppcfloor/comm/sys/lib -ldcmf.cnk      -ldcmfcoll.cnk      -lpthread -L/bgsys/drivers/ppcfloor/runtime/SPI -Wl,-rpath,/bgsys/drivers/ppcfloor/runtime/SPI -lSPI.cna -lrt" \
../configure \
--enable-cache \
--prefix=/opt/apps/gcc/4.3.2/comm/default \
--mandir=/opt/apps/gcc/4.3.2/comm/default/man \
--htmldir=/opt/apps/gcc/4.3.2/comm/default/www \
--host=powerpc-bgp-linux \
--target=powerpc-bgp-linux \
--build=powerpc64-linux-gnu \
--enable-dependencies \
--enable-debuginfo \
--enable-g=none \
--enable-sharedlibs=gcc \
--with-device=dcmfd:BGP \
--enable-romio=yes \
--enable-f77=yes \
--enable-f90=yes \
--with-file-system=bgl+bglockless \
--enable-timer-type=device \
--enable-threads=multiple \
--with-fwrapname=fmpich.cnk \
--with-cross=../src/mpid/dcmfd/cross8 \
--with-mpe=no \
--enable-wrappers \
--disable-fast \
--enable-timing=no \
--disable-collchk \
--disable-graphics \
--disable-rlog \
--disable-sample \
--with-assert-level=2 \
--with-aint-size=8 \

make && make install

VOILA. You have a new MPICH installed in /opt/apps/gcc/4.3.2/comm/default

Masala Koora

August 27th, 2013 by kkt3863

 

Ingredients

  • 1/2 Large Red Onion
  • 1 tomato
  • 1 tbsp ginger paste
  • 1 tbsp garlic paste
  • 1 tbsp sambar powder
  • 1 tsp poppy seeds
  • 1 tbsp coconut powder or 1 small chunk fresh coconut
  • 2 almonds
  • 1 tbsp mustard seeds
  • 1 sprig curry leaves
  • Your favorite vegetables
  • Salt to taste

Blend 1/2 an onion, tomato, almonds, sambar powder, ginger & garlic pastes, poppy seeds and coconut to make the masala paste.

Begin tempering by heating oil and adding mustard seeds and curry leaves. When the mustard starts to sputter add the remaining finely chopped onion. Add the masala paste and cook for a little while until the raw onion smell is gone. Add vegetables (peas/garbanzo beans/whatever) and fry in the mixture. Cook on medium/low flame until the vegetables are cooked.Optional: Transfer the mixture into a pressure cooker and cook until 1 whistle.

Compiling Quantum Espresson on Linux PPC64

January 23rd, 2012 by kkt3863

1. Download; untar and configure

./configure FFT_LIBS="-L/opt/apps/fftw/3.2.2/lib -lfftw3f" BLAS_LIBS="-L/opt/apps/ibm/essl/5.1/lib64 -lessl" LAPACK_LIBS="-L/opt/apps/ibm/essl/5.1/lib64 -lessl -llapack" LIBDIRS="/opt/apps/fftw/3.2.2/lib /opt/apps/ibm/essl/5.1/lib64/" MPIF90=mpif90 CFLAGS="-O3 -q64" FC=mpif90 F77=mpif90 F90=mpif90 CC=mpicc FFLAGS="-q64" --enable-parallel

2. Edit make.sys and replace F90, F77, CC with xlf90_r, xlf_r and xlc_r respectively.
3. make all

Banana Cupcakes with Caramel Cream Cheese Frosting

January 21st, 2012 by kkt3863

Thanks to my colleague, Clinton Heider, for sharing his family recipe.

Ingredients

2 ¼ cups flour
½ teaspoon baking powder
¾ teaspoon baking soda
½ teaspoon salt
½ cup soft butter
1 ½ cups sugar
2 eggs
1 cup mashed bananas
1 teaspoon vanilla
¼ cup buttermilk

Cupcake

Prepare cupcake pans with paper liners.

Sift flour with baking powder, baking soda and salt.

In a separate bowl, mix banana with vanilla and buttermilk.

Cream butter, then add sugar and cream together until light and fluffy. Beat in eggs, one at a time. Add the flour and banana mixtures in parts, alternating and blending completely after each addition.

Fill baking cups with batter and bake for 15 – 20 minutes, until toothpick comes out clean.
Cool completely before frosting.

Caramel Cream Cheese Frosting

8 oz cream cheese, chilled
1 stick butter at room temperature
½ cup caramel sauce
3 cups powdered sugar

Blend cold cream cheese with butter until well blended, then add caramel sauce and blend until incorporated. Slowly add powdered sugar, blending until incorporated.

VNC port forwarding over SSH

January 19th, 2012 by kkt3863

1. Start vncserver on the remote machine:
vncserver

Check which display the server is running on:
[user@login1 ~]$ vncserver -list

TigerVNC server sessions:

X DISPLAY # PROCESS ID
:2 9416

In this case, its running on display “:2”

Also check which port the VNC server is running on:
netstat -tulpn

2. In a terminal on your desktop, start SSH forwarding:
ssh -L 5902:localhost:5902 -N @remoteserver

Keep this session open until you want your VNC port active.

3. Connect vnc client to the SSH forwarded port:
vncviewer localhost:2

Android on HP Touchpad

October 18th, 2011 by kkt3863

As of today, Cyanogenmod 7.1 is still in Alpha, but still usable on the HP Touchpad. I still haven’t installed it yet; possibly try it in a few days. PC Pro has a step by step installation guide for the unadventurous.

http://www.pcpro.co.uk/features/370558/how-to-install-android-on-the-hp-touchpad

Linking with multiple definitions of symbols

April 1st, 2011 by kkt3863

Add the following flags -Xlinker -zmuldefs to the compiler.

Creating a Shared Library from a Static Archive

March 9th, 2011 by kkt3863
ar -x libatlas.a
gcc -fPIC -shared *.o -o libatlas.so

If you encounter an error with the linker that refers to multiple definitions of a function then pass the following option to the linker:

gcc -Xlinker -fPIC -shared *.o -o libatlas.so

Build ATLAS on Linux

March 9th, 2011 by kkt3863
  1. Get the latest version of ATLAS from: http://sourceforge.net/projects/math-atlas
  2. Untar and get into the source directory.
  3. Follow these steps:
mkdir BUILD
cd BUILD
../configure -b 64 -Fa alg -fPIC --with-netlib-lapack-tarfile=<path to lapack.tgz> --prefix=/home/sam/local
make
cd lib
make shared
make ptshared
cd ..
make install