Thursday, December 06, 2012

How to Shrink, Compress, Backup DVDs

This post shows a number of ways and tools to accomplish the following:
Goal: To backup a DVD which is originally more than 4GB onto one standard single DVD about 4GB. The end-product needs to be the single DVD playable on most DVD players, not just playable on PCs.

When: This can happen when your Wedding DVD, your Graduation DVD or some of your Personal DVD happen to be in a DVD with all the contents totalling, say 6GB.

Here are the different parts to convert and backup your Personal DVD into a single playable DVD. The actual process of conversion and backup is described further down.

A. Open DVD and Save to ISO
Use DVD Decrypter 3.5.4.0.
Load the DVD into the drive.
Open the Disc from DVD Decrypter.
Choose the Source as the Drive
Choose the Destination as a file.
Run the program.

B. Open DVD and Save to Compressed  ISO
Use DVD Shrink 3.2
Load the DVD into the drive
Select whatever VOB files and other Files.
Run Backup

C. Open 6GB ISO file and Save to Compressed  ISO
Use DVD Shrink 3.2
Load the ISO file in DVD Shrink
Select whatever VOB files and other Files.
Run Backup

D. Open 6GB ISO file as a Drive
Use MagicDisc from Magic ISO.
Choose the 6GB iso file and MOUNT using MagicDisc.
Now a virtual DVD drive appears with the contents of the 6GB file.

E. Burn Compressed ISO 4GB to single DVD
Use ImgBurn as the DVD burner.
Choose the Source as the 4GB compresses ISO image.
Choose the Destination as the real drive with blank DVD.
Burn ISO to DVD.

The actual process can include the following steps.
Option 1:
   A  ->  C  ->  E
Option 2:
   B  ->  E
Option 3:
   A  ->  D  ->  B  -> E

What I found is that there are many ways to do this. But it appears that option 2, may be the most efficient. Option 2 uses DVD Shrink to read your original 6GB DVD and compress to 4GB standard DVD ISO file. Then ImgBurn is used to burn the image.

Extras:
- There is no need to consider audio or DVD menu since they will be automatically included.
- ISOBuster 1.5 is another tool that allow you to examine the contents of the ISO file.
- VirtualBox - within the virtual machine, there is the capability to mount an ISO file as a virtual DVD drive.

Thursday, November 01, 2012

Notes on Windows

NotesWindows  - Notes on Windows
=============

This includes NotesWinXP which is now made obsolete.
The content here is related to any version of Windows and will be specifically denoted.


Contents
=========
Roaming Profile - Encryption
Useful Registry Keys
Stop accidentally moving files or folders
Windows Update does not work anymore
Windows CMD commands
Application Location
MS Windows XP backup
Change Location of My Pictures, My Documents
Change Location of Libraries in Windows 7
Password Revealed on Windows
Known Folders can be Hidden - Windows 7
Boot Manager - Windows XP, Windows 7
How to Repair a Corrupt Windows 7 Installation
Windows Recovery Console Operations
Windows 7 Administrator account
Windows 7 - Where are those Options
Windows 7 - Create a System Repair Disc
Microsoft Outlook
Windows 8 - Confirm Delete
Windows 7/8 Install - Creating Bootable USB
Windows 8 - Startup
Windows 8 - Startup Folder
Removing Rogueware
Hidden Windows Devices


Roaming Profile - Encryption - WinXP
=============================
Problem: When roaming profile cannot be saved - the reason could be due to some files are being encrypted.

Solution: Find and remove the encrypted file. Use the tool "cipher" provided by WinXP.

    cipher /S:directoryName
... to check encryption status of all subdirectories

Ref:
Roaming decrypt
http://www.microsoft.com/technet/prodtechnol/winxppro/reskit/c18621675.mspx

Search windows "roaming profile" decrypt
http://au.alpha.yahoo.com/search/web?p=windows+%22roaming+profile%22+decrypt
search http://au.alpha.yahoo.com/search/web?p=windows+%22roaming+profile%22+decrypt


 

Useful Registry Keys
================================
Append Path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths
System Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment
User path:   HKEY_CURRENT_USER\Environment
Reserved Drives Letters: HKEY_LOCAL_MACHINE\SYSTEM\MOUNTEDDevices

Stop accidentally moving files or folders  - WinXP
===========================================
It is very easy to accidentally move folders in Windows Explorer, simply by careless movement of the mouse and accidentally releasing buttons. This problem is also very easy to occur with trackpads on laptops. There is a way of preventing this accidental motion by making Windows pop up a dialog box to ask for confirmation.

"Do You Want To Move Or Copy Files From This Zone?"

To enable this feature,
1. Click Start -> Control Panel -> Internet Options.
2. Click “Security” tab, mark the zone “Local intranet”, and click the “Sites” button.
3. Un-tick “Automatically detect intranet network”
4. Tick “Include all local (intranet) sites not listed in other zones”
   Tick “Include all sites that bypass the proxy server”
   Untick “Include all network paths (UNCs)”  

Windows Update does not work anymore  - WinXP
=====================================
For some reason, Windows files may corrupt such that the Windows Update will not work anymore. To solve this:
1. Open a Command Prompt Terminal: Start - Run, type: cmd.
2. To stop the current windows update service, in the Command Prompt, type:
    net stop wuauserv
3. Re-register the windows update dll for WinXP 32bit
    regsvr32 %windir%\system32\wups2.dll
Re-register the windows update dll for WinXP 64bit
    regsvr32 %windir%\syswow64\wups2.dll


Windows CMD commands
=====================
Ref: http://ss64.com/nt/

When in a CMD terminal, to launch another CMD terminal and execute a script (test.bat)
- start cmd /k test.bat    - new window remains open
- start cmd /c test.bat    - closes the new window when script finishes

Some commands related to Volume Shadow Copy Service (Vista)
vssadmin List Shadows
vssadmin List ShadowStorage


Application Location
=====================
WinXP:
Netscape Bookmarks (Netscape v 8.1)
D:\Program Files\Netscape\Netscape Browser\defaults\profile\bookmarks.html


MS Windows XP backup
======================
Below describes the process to take when backing up Windows. Some folder directory sizes given here may give an indication of their relative or typical sizes, but they will obviously vary in different systems.

- Login as each user and i) Empty Recycle Bin, ii) Clear all Cache, Cookies
- Login in as Administrator (so that can backup Registry and other System State)
- Goto Control Panel - Performance and Maintenance - "Free up space in your hard disk"
- c:\windows\system32 = 1GB
- c:\windows\ folders except system32 = 790MB
- c:\windows\system32\dllcache = 467MB
- c:\windows\$NTUninstall* = 245MB
- c:\windows\Driver Cache = 87MB
- c:\windows\fonts = 89MB
- c:\windows\help = 46MB
- c:\windows\ime = 86MB
- To backup Registry - go to the GUI windows Backup Utility and check the "System State"
- Do Normal and then Incremental backup.


Change Location of My Documents
=================================
WinXP:
Change My Documents.
- Right click My Documents - Properties - change the Target Location
- by default My Documents is in C:\Documents and Settings\<user>\My Documents

My Pictures, My Music.
- It appears that these folders are not changeable and are set under My Documents.
- When My Documents location is changed, then these folders should follow too.


Change Location of Libraries in Windows 7
===========================================
Libraries in Windows 7 can be assigned to various locations. Libraries will source the physial location of target directories and present the contents as if they all belong to the library. To change the location of Libraries in Windows 7:
- open Explorer
- in the left hand pane, right click on the library icons
- select Properties
- a list of location is presented, which can then be modified


Password Revealed on Windows
==============================
Windows 7:
- Passwords apparently can be revealed by System Information for Windows.
- the System Information (Start - All Programs - Accessories - System Tools - System Information) on Windows XP do not seem to reveal passwords.
- to remove the passwords in Windows 7, do:
    Control Panel - User Accounts and Family Safety - Windows Credential Manager.



Known Folders can be Hidden - Windows 7
==========================================
Windows default folders like Music, Pictures, Videos, Searches are part of the operating system and is not easy to remove. They may be hidden in a number of different ways.
1. Move them to another location by:
- Right click and select properties of the Known Folder
- Go to Location tab
- use the button Move to specify new location.
2. Merge them into Documents folder.
- Select the Known Folder to move.
- For the target to move to, select Documents.
3. Remove Known Folders
- Go to Start button and type, GPEDIT.MSC
- Under User Configuration, click Administrative Templates, double click Windows Components, then Windows Explorer.
- In the Disable Known Folders setting, double click and select Enabled.
- Click Show and enter names of any folders to be disabled.
- Go back to main Windows Explorer and delete any of the Known Folders, and it will be permanenet.

For a list of Known Folders, http://msdn.microsoft.com/en-us/library/bb882665.aspx


Boot Manager - Windows XP, Windows 7
======================================
Boot Loader for Windows XP: NTLDR
Sits on the Primary Partition of the Hard Disk
Configuration file: boot.ini
To edit, use Notepad

Boot Loader for Windows 7 / Vista : Windows Boot Manager (BOOTMGR)
Sits on the Primary Partition of the Hard Disk
Configuration file: BCD (Boot Configuration Data)
To edit, use BCDEDIT


How to Repair a Corrupt Windows 7 Installation
================================================
http://www.computerworld.com/s/article/9221540/How_to_Repair_a_Corrupt_Windows_7_Installation?taxonomyId=89&pageNumber=3


Windows Recovery Console Operations
====================================
Firstly to enter into recovery console, by booting from Windows CD/DVD. When 'Welcome to Setup' appears, then press 'R'. Here is a list of commands that may be helpful for repairing or trouble shooting in Recovery Console mode.

listsvc        - list the services running. Example display drivers agp440.sys is listed
disable agp440 - to disable a service that is running and listed under listsvc.


Windows 7 Administrator account
==================================
This section is specifically for Windows 7. In previous versions of Windows, there may appear the Administrator user on the login page. The Administrator account does not appear on Windows 7 by default.

By default, the Windows 7 installation usually
- allow you to create a new user, say FirstUser, during install and will set it up with Administrator password.
- there is a hidden Administrator user, but it is not active and has no password.

To manage the Administrator account, eg to activate it or give it a password, etc:
- click Start
- right click Computer - Manage
- Go to System Tools - Local User and Groups - User
- perform any task including Administrator account.

Alternative way to make Administrator appear on login screen.
- click Start and in the Search box, type CMD
- right click on the CMD icon that appears, choose Run As Administrator
- in the Command Prompt type:
  net user administrator /active:yes
      or
  net user administrator /active:no


Windows 7 - Where are those Options
=====================================
To enable Folder Options, which used to be a menu within Windows Explorer, we now need to do this for Windows 7:
- go to Contorl Panel - Appearance and Personalization - Folder Options.


Windows 7 - Create a System Repair Disc
========================================

The GUI way to create a Repair Disc is to Saerch for "Create a Repair Disc"

However, that can fail like this: once click create button and error came out
    original title: the parameter is incorrect ( 0x80070057)
    Unable to create a system repair disc 0x80070057

In that case, an alternative to create Repair Disc is:

i.           Boot the system by inserting Windows 7 installation disc.
ii.         When it gets to the main setup window, click on Repair from the lower left corner.
iii.       Select Command Prompt from the main Repair menu.
iv.        At the command prompt type in recdisc, then ENTER.
v.          You'll get the popup to choose which drive letter to create the Recovery disc to.
vi.        Have a clean DVD in the DVD drive and select start.
This should get your System repair disc created.



Microsoft Outlook
==================
Before you say this section should not be in the Windows notes, we argue that it should simply because MS love to tie in the products to the Windows OS so intimately that it affects other Windows related stuff, as you will see.


How to save, manage, archive Microsoft Outlook *.msg files?

- If your messages are in the Notes folder, or if you are looking for Calendar, Inbox or other items, DON'T LOOK FOR NOTES folder. Everything in Outlook inside Inbox, Notes, Calendar and more are DUMPED into a single *.pst file.
- - Where is the *.pst files?
- - - In Windows 7, it is default in:
- - - C:\Users\cyk\Documents\Outlook Files\Outlook.pst     (main outlook file)
- - - C:\Users\<user name>\AppData\Local\Microsoft\Outlook\*.pst   (other email accounts imported into Outlook)

- Tricky, you may be smart and Open Outlook, go to the left hand panel and find Inbox, Notes. Then right click on Notes, select properties. This will show the Notes folder in location called: "Outlook Data File". Don't spend hours looking for this, it is in fact the Outlook.pst mentioned above.

- So you cannot access all your messages without using Outlook, because it is also dumped into the Outlook.pst which is a proprietary format. Here's how to get all the messages out.
- - Open MS Outlook.
- - Open a Windows Explorer window and place it next to the Outlook app.
- - From the Outlook, click Notes, click the big buttons in the Ribbon to change to "Notes List". This give you a list of all the Notes in the main pane of Outlook.
- - Highlight all the notes you can to archive and save elsewhere. Drag them to a folder in the Windows Explorer window. Now all your notes will appear as individual *.msg files.
- - Download the little utility called "msgtxt" from:
http://www.enterag.ch/enterag/downloads/msgtext.xhtml
http://www.softpedia.com/get/Internet/Other-Internet-Related/MsgText.shtml
- - Use msgtxt to convert *.pst to *.txt files. A simple batch program can be written to simplify this.



Windows 8 - Confirm Delete
=============================
In Windows Explorer in Windows 8, it seems that it doesn't ask for confirmatin when you delete something.

Is this crazy or what? Not sure about those people who insists they don't make mistakes, but it is much safer to have the Confirm Delete option, which have been present in Windows many generations ago. So Windows 8 decide to trash this safety feature.

But you can bring it back - NOT from the Folder Options area.

You need to go to Windows 8 Desktop (no, not from the Metro UI), right click Recycle Bin.
Then click on Properties.
Then check the box 'Display delete confirmatin dialog'

If you want to test this, don't try deleting any important files ;)
Choose a dummy file that you don't mind losing and test this feature is back or not.


Windows 7/8 Install - Creating Bootable USB
=============================================
Creating the USB drive is the first step to installing Windows 7/8 from USB.

UEFI vs BIOS
For UEFI, the USB needs to be formatted as FAT32
For BIOS, the USB needs to be formatted as NTFS

When creating the USB on another PC, to check if that PC is booted with EUFI or not, launch "MSINFO32" on the CMD, then look at the BOOT MODE.

Size of USB drive needs to be at least 4GB


- Open a Command Prompt with Administrator privileges in Windows 7/8.
- Insert the USB device into an available USB port.
- Type "DiskPart" in the command prompt.
- Type "List Disk” (make note of the disk number of the target USB drive).
- Type "Select Disk X”, where X is the target USB drive noted in step above.
- Type "Clean”.
- Type "Create Partition Primary”.
- Select Partition 1
- Type "format FS=fat32 quick” for UEFI installation
    OR
  Type "format FS=ntfs quick” for BIOS installation
- Type "Active”.
- Type "Assign".
- Type "list volume".
- Type "Exit".


Extra step for UEFI
If the file 'BOOTX64.EFI' is not in \EFI\BOOT\ on the stick then copy the file to the stick by:
- go to a UEFI PC with Windows 7/8 x64, copy from C:\Windows\Boot\EFI for the file 'bootmgfw.efi to \EFI\BOOT\BOOTX64.EFI on the USB.


May NOT need to use Bootsec
- Mount the Windows ISO and access the ISO contents.
- cd to "boot" subdirectory
- type: bootsect.exe /nt60 X: where X is the drive letter for the USB key.
- this infuses boot manager code into USB key to make it bootable.


Ref: http://support.lenovo.com/en_AU/detail.page?DocID=HT076615
https://blogs.technet.com/b/home_is_where_i_lay_my_head/archive/2012/10/02/how-to-install-windows-7-or-8-from-usb-using-uefi.aspx?Redirected=true
http://www.thomas-krenn.com/en/wiki/Creating_Windows_UEFI_Boot-Stick_in_Windows
http://answers.microsoft.com/en-us/windows/forum/windows_7-windows_install/problem-using-win7-usbdvd-download-tool-concerning/37eadf15-85dd-4e56-a89c-fa0a3339b92e
http://www.maximumpc.com/article/howtos/how_to_install_windows_7_beta_a_usb_key



Windows 7/8 Install - Installing from USB
=============================================

UEFI vs BIOS
With new Windowss and new Motherboards there is an option to use BIOS or UEFI.
Please consider this first.


1. Prepare a USB by first making it bootable, see - "Windows 7/8 Install - Creating Bootable USB"

2. Access the installation DVD for Win7/8.

3. Dump all the contents of the DVD onto the USB drive.
xcopy SRC_DRIVE:\* DST_DRIVE:\ /s /e

4. Use the USB to install on a new machine.


Windos 8 - Startup
===================
http://www.howtogeek.com/126016/three-ways-to-access-the-windows-8-boot-options-menu/

UEFI and Windows 8 Update on Windows/Linux dual-boot systems
http://www.zdnet.com/uefi-and-windows-8-update-on-windowslinux-dual-boot-systems-7000028217/

Seven ways to set up multi-booting with Windows 8 and Linux
http://www.zdnet.com/seven-ways-to-set-up-multi-booting-with-windows-8-and-linux-7000026392/

How to dual boot Windows 8 and Linux
http://apcmag.com/how-to-dual-boot-windows-8-and-linux.htm


Windows 8 - Startup Folder
=============================
Unlike previous versions of Windows, the Windows 8 startup folder is typically located at:
         C:\Users\<username>\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

Put any *.bat files here to make it run automatically at startup


Removing Rogueware
===================
- create restore point
- Use the following tools to scan and remove rogueware
AdwCleaner (www.bleepingcomputer.com/download/adwcleaner) to run scan
Junkware Removal Tool (www.bleepingcomputer.com/download/junkware-removal-tool) and run scan.
RogueKiller (www.bleepingcomputer.com/download/roguekiller) and run scan

Hidden Windows Devices
=======================
Some USB devices which have been unplugged or not active cannot be seen via Windows' Device Manager. However the following command will make Windows show the Hidden Devices. Open a CMD terminal, and enter the two lines:
- set devmgr_show_nopresent_devices=1
- start devmgmt.msc

Thursday, August 30, 2012

Windows Server 2008 - Firewall, Antivirus, AntiSpyware

This article list some free, cheap or affordable alternatives for security software for Windows Server 2008 and possibly other Windows Server family of products. I was initially surprised to find that popular security products including Firewall, Antivirus and AntiSpyware do not work for Windows Server 2008, but worked for Windows 7. Especially due to the fact that Windows 7 and Windows Server 2008 share the same code base, this means they have the same engine under the hood.

Yet the Firewall and Antivirus and AntiSpyware that worked on Windows 7, refused to work on Windows Server 2008 and I think I know the reason. The reason must be purely commercial. Look at some software from Avira and Emsisoft, their Windows Server products are classified as Business Products but the Windows 7 and Vista are Home products. The Home products are between $50 - $100 while the Server or Business products can be more than $300.


So here are the things I tried and have not worked - so you can save yourself the trouble. Then I will show the things that do work.

Firewall that Don't work on Windows Server 2008
Comodo Firewall
Outpost Security Suite Free

Antivirus that Don't work on Windows Server 2008
Avira Antivirus
Avast Antivirus

Emsisoft Trial did work - but I'm not sure if it will work if I buy the standard edition. There is a server edition that costs hundreds of dollars.

Now for the good news.


Firewall that Work on Windows Server 2008
Zone Alarm Antivirus and Firewall Free

Antivirus that Work on Windows Server 2008
Zone Alarm Antivirus and Firewall Free
Microsoft Security Essentials
WinPatrol
SuperAntiSpyware
Spybot Search and Destroy
ThreatFire
Spyware Terminator

For Reviews, Discounts and Free Security Software, see:
http://xtechnotes.blogspot.com.au/2012/01/security-software-discounts-and-special.html




Saturday, August 11, 2012

How To Point Your Blogger - Blogspot Blog to Your Own Domain Name


The following steps show you how to connect your existing blog to your own domain name.

Why do this?
- You have a blog about BLAH already, at www.blah.blogspot.com or blah.blogspot.com
Notice the "blogspot" word which seems rather messy.
- So you actually want something like:
www.blah.com    or www.blah.org
..... then read on ......

What you need?
- You already have established a blog with Blogger, also called Blogspot.com
- You need to buy / register a Domain Name.
- Having a Domain Name is not enough, you also need to get DNS services from the Domain Name seller. This is an extra charge.

What is DNS Services?
- When you see things like "CNAME" or "MX records" or "C records", you are on the right track.
- From the CrazyDomains website, DNS services allow you to:
"Modify the zone files for your DNS information assigned to your domain. Update and add A records, CNAME or MX records."
- In simple terms, it lets you point your www.blah.com website to the actual blog at blah.blogspot.com. When people go to the www.blah.com, they will see your blog.

How to do it?
1. Ensure all the requirements above are met.
2. Get login DNS settings page for the domain.
3. Edit the domain settings to add a record:
   Fully Qualified Domain Name: www
   Record Type:CNAME
   Record Value: ghs.google.com
4. Go to Blogspot.com and login.
5. Go to "Settings", then "Basic", then "Publishing".
6. Below the "Blog Address", click the link "Add a Custom Domain".
7. Click "Switch to Advanced Settings".
8. Follow the instructions on the screen to complete.

Others
http://kb.weebly.com/domain-settings.html#arecord
The link above talks about MX, A-Records, CNames and how they are used when the Hosting And Domain provider are different. Very useful in setting up new domains or new web hosting different to domains.

Saturday, July 21, 2012

Days of the Week - Explained

This is a quick method of how to find the Day of the Week given the Date, Month Year

The steps are:




1. Take last 2 digits of year, take '92' for 1992
2. multiply by 5 -> 92*5
3. divide by 4   -> 92*5 / 4
4. erase decimal -> remove the decimal in the previous result
5. add month code
6. add date
7. divide by 7, take the 1st decimal point and use the Day Code below to look up the Day


Month Code
1 j
4 f
4 m
0 a
2 m
5 j
0 j
3 a
6 s
1 o
4 n
6        d


Day code

1 Sun
2 Mon
4 Tue
5 Wed
7 Thu
8 Fri
0 Sat

Wednesday, July 18, 2012

GSLWintel



GSLWintel
===========
Presented here is the GSL (GNU Scientific Library) compiled with Intel C/C++ on Windows
The GSL 1.15 was build with Visual Studio 2010 (VS), using Intel C/C++ 12.1 (Composer XE 2011 SP1).
The GSLWintel complete package contains one VS Solution (gslIntelSoln) and multiple VS Projects.
The GSL 1.15 source code has been placed in the appropriate directories. You can download the GSL source code and extract it to some other location for the purpose of comparing the source code with GSLWintel.
Upgradeble to GSL 1.16 and beyond - in order to do this, follow the build steps in "Building using Intel C/C++"
Absolute minimal changes to original GSL source code - all the hard work has been done in constructing the VS Solution and projects to consume the source code. To see the changes, see "Building using Intel C/C++" section.


Notation
=========
<gslsrc>     - the original gsl source tree
gslIntelSoln - the VS solution that holds this gsl build
gslIntel     - the name of the main gsl project under gslIntelSoln


What's different from between GSLWintel and Standard GSL on Linux
====================================================================
- this is build for Windows, 32bit or 64 bit.
- uses Intel C/C++ compilers
- for BLAS routines, the MKL library is used instead of the GSL cblas package. The reason is that since the Intel Compiler is being used already, the user should already have access to Intel's optimized blas library called MKL.
- <gslsrc>\cblas package is not included since MKL blas will be used.
- <gslsrc>\doc package is not included
- <gslsrc>\utils package is included but not compiled. Does not seem to have adverse affects
- to see what's missing or different from the original gsl source, you can always download the gsl source from the main GSL site, extract it to a different folder, and use it as a reference for comparison.


System Requirements
=====================
Visual Studio 2010
Intel C/C++ Compiler 12.1 with Intel MKL
Windows XP or later
GSL 1.15 source code


GSLWintel Structure
====================
- GSLWintel complete package contains one VS Solution called gslIntelSoln at the top level
- Under gslIntelSoln, there is VS project called gslIntel. This contains all the original gsl source code to be compiled into one GSL dynamic library called gslIntel.dll and gslIntel.lib.
- Under gslIntelSoln, there are also multiple VS test projects corresponding to each gsl package. For instance, gsl packages are block, complex, eigen, etc. Then there will be VS projects called test_block, test_complex, test_eigen.


Installation and Build
=========================
- The GSLWintel package is available on-demand. Please request it by posting a comment below.
- Extract the GSLWintel package and check (using Windows Explorer) that the structure is the same as described in the section above "GSLWintel Structure".
- Ensure Intel C/C++ compilers and Visual Studio 2010 are installed on your system.
- Open the solution gslIntelSoln.
- GSLWintel has been successfully build and tested for 32bit and 64bit.
  For 32bit choose Solution Configuration / Platform as: Debug / Win32
  For 64bit choose Solution Configuration / Platform as: ReleaseX64 / x64
- Simply Build the gslIntel VS project first.
- Then simply Build the other test_<package> VS projects. The test projects can be build one by one to see how it goes. Or highlight multiple test_<package> in Solution Explorer, then click Build Selection.
- Output is one gslIntel.lib, one gslIntel.dll. Also test_<package>.exe such as: test_block.exe, test_complex.exe
  For 32bit, gslIntel.* output is in   gslIntelSoln\Debug\
             test_<package>.exe are in gslIntelSoln\Debug\
  For 64bit, gslIntel.* output is in   gslIntelSoln\x64\ReleaseX64\
             test_<package>.exe are in gslIntelSoln\x64\ReleaseX64\
- GSLWintel is designed to be rebuilt easily by putting into the specific VS solutions and projects. But as always, unexpected errors will happen. In that case, you may need to debug yourself, but before doing that, read the section "Building using Intel C/C++" to gain an understanding of how GSLWintel was designed. Also see the Troubleshooting section.



Building using Intel C/C++
============================
Read this section if you want to understand how GWintel is put together.
The instructions in this section can be used to build your own GSL - Visual Studio 2010 Solutions and Projects.
This will be necessary if you wish build GSL 1.16 or any future versions.


This is based on the strategy used for "Building using Visual Studio C/C++"
Major structural change - using method from : http://gladman.plushost.co.uk/oldsite/computing/gnu_scientific_library.php

However, for the reasons below, we will not use gladman's template directly but instead use the same concept. The reasons are:
1. Using Gladman's VS2010 templates verbatim, may not work for the next version of GSL. It is better to build from scratch, documenting the steps, rather than using a black box.
2. Gladman's approach uses some processing of GSL source, and again, this may not work for future versions.
3. There is no guarantee of reduced effort if using Gladman's code, since a bug may take longer time to resolve.



Creating gsl project. For convenience, let:
<proj> = "gslIntel",
<gslSrc> = "src_1.15" = where original GSL source is extracted.


1. Prepare the following directory structure using Windows Explorer:
GSL_staging
    gslIntelSoln
    <proj>     (created by VS automatically)
<gslSrc>

Extract the raw / original GSL1.15 into <gslSrc>


2. Add new C++ project, with name <proj>.
Choose Location as ...../GSL_staging
Choose Solution name as gslIntelSoln
Choose "Create directory for solution"
Choose "Win32 project". Choose Application Type as "DLL", "Empty Project"

3. Convert to Intel C++ project. Right click on the project "gslIntel" in Solution Explorer.
Project -> Intel C++ Composer XE 2011 -> Use Intel C++

4. Go to the properties of <proj> and set the following properties:
Configuration Property
   C/C++
      General
    Suppress Startup Banner = NO     (this identifies which compiler is used).
//Additional Include Directory = $(ProjectDir)     ( this is for config.h)

5. Adding GSL source files. This section copies the GSL source files into the appropriate VS-project directories.
Manually Copy the files as shown below, note the name changes for some.
FROM GSL_staging/<gslSrc>                           TO  GSL_staging/gslIntelSoln/<proj>
   <package folder>/*.c                                  <package folder>/*.c
   <package folder>/*.h                                  <package folder>/*.h
   <package folder>/gsl_*.h                              gsl/gsl_*.h
   gsl_<package>.h                                       gsl/gsl_<package>.h
   config.h.in                                           config.h
   templates_on.h                                        templates_on.h
   templates_off.h                                       templates_off.h
   build.h                                               build.h

- Under GSL_staging/gslIntelSoln/<proj>, Solution Explorer has TWO VIEWS:
a. Show All Files ON
   This shows all subdirectories and all files, and most files are NOT included in the project to start with.
b. Show All Files OFF
   This shows the following FOUR folders, which will be filled automatically as files are Included into the project. The folders are:
   --- External Dependencies
   --- Header Files
   --- Resource Files
   --- Source Files
c. <package folder> can be block, err, matrix, etc....


d. To add the following packages into VS Projects, eg block, err, matrix, etc.....
- In Solution Explorer, choose Show All Files ON
- Go to each package sub-directory and add source files BY select "Include in Project". Include any other project as necessary, only include their *.c is OK. All header files do not need to be INCLUDED via Visual Studio because the C compiler will pull the contents of *.h hearder files in.
The general rules are:
----- Include *.c files mainly. Some included like *_source.c or *_inc.c or test_*.c and other test related files
      are also INCLUDED in Project, but EXCLUDED from Build.      
 Individual files can be excluded from Build (ie won't compile) by setting their properties individually.
      In Solution Explorer, choose Show All Files OFF. Go to "Source Files" directory to confirm package files are there.
 Also <package>\*.h files that do not begin with "gsl_".h, must also be included in the project.
----- Exclude <package>\gsl_*.h from project. These will be copied into the gsl sub-directory.

Build files. Assuming for Debug configuration, by default, *.obj files will be placed under:
      GSL_staging/gslIntelSoln/<proj>/Debug
By Default, *.dll will be placed in:
      GSL_staging/gslIntelSoln/Debug

WARNING: Some files, eg. file.c exist in multiple packages. The following step has to be done on file by file basis.
For most files which have unique names, ensure the compiled output is in:
- Project Properties - C/C++ - Output Files - Object File Name : $(IntDir)
For files like file.c with same name but in many packages, ensure that:
- Project Properties - C/C++ - Output Files - Object File Name : $(IntDir)<package>\
  where <package> is the name of the package
Example: since file.c is in packages histogram, matrix, etc, the property would be
  $(IntDir)histogram\
  $(IntDir)matrix\

To be complete, here is a list of *.c files that exist in multiple directories:
balance.c
beta.c
beta_inc.c
binomial.c
brent.c
bsimp.c
bspline.c
cauchy.c
chisq.c
combination.c
control.c
convergence.c
copy.c
cstd.c
deriv.c
diff.c
dwt.c
exppow.c
fdfsolver.c
fdist.c
fft.c
file.c
flat.c
fsolver.c
gamma.c
gauss.c
geometric.c
hyperg.c
init.c
inline.c
interp.c
laplace.c
linear.c
logistic.c
lognormal.c
matrix.c
minmax.c
multiset.c
nbinomial.c
newton.c
oper.c
pareto.c
pascal.c
poisson.c
prop.c
qr.c
qrng.c
rayleigh.c
reset.c
rk2.c
rk2imp.c
rk4.c
rk4imp.c
rk8pd.c
rkck.c
rkf45.c
rng.c
siman.c
sort.c
stat.c
step.c
swap.c
tdist.c
vector.c
view.c
weibull.c

*** source which are not compiled, eg *_source.c, *_inc.c, test* are ignored.



6. Edit the config file so that the "!" is removed in the following lines:
#if HAVE_DECL_LDEXP
#if HAVE_DECL_FREXP
#if HAVE_DECL_HYPOT
For some reason the use of preprocessor macros does not seem to work well. Hence the config.h file need to be modified as above.


7. In many matrix files, eg copy_source.c, file_source.c, getset_source.c, the copy.c, file.c, getset.c uses the templates_on/off.h which dynamically creates code using Preprocessor Macro definitions. This causes SEVERE problems at the line:
#define SHORT complex
because Microsoft Visual C redefines this in their math.h file as:
#define complex _complex
DO  NOT Put this macro definition /D__cplusplus  which may solve the _complex problem but creates new problems with other MS files like:

Error 1 error C2061: syntax error : identifier 'vc_attributes' c:\program files\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h 42 VCPP_testDLL01

To solve the _complex problem definition, add this at the top of templates_on.h:
#ifdef complex
#define fixCOMPLEX_ 1
#undef complex
#endif

Also add this at the end of templates_off.h
// ASSUMES that the predefined complex is _complex in Microsoft's VC include\math.h and crt\src\math.h
#if fixCOMPLEX_ == 1
#define complex _complex
#undef fixCOMPLEX_
#endif

8. Right click on the GSLProj icon and select properties. In the Properties configuration, Configuration Properties:
- under C/C++ - Advanced
Calling Convention -  __cdecl (/Gd)
    Compile As         - Compile as C Code (/TC)
- under C/C++ - Preprocessor
    these should be automatically created here: WIN32, _DEBUG, _WINDOWS, _USRDLL, <Project>_EXPORTS
    manually added: GSL_DLL, DLL_EXPORT
- under C/C++ - Command Line
Add these compiler options: /DGSL_DLL /DDLL_EXPORT /DHAVE_VPRINTF=1
- under C/C++ - General - Additional Include Directories:
      enter the path to the current project folder.
- under Linker - General - Additional Library Directories:
      enter <fortranCompiler>\fortran\lib\ia32
- Go to the <proj> Properties - Configuration Properties - Intel Performance Libraries - Intel Math Kernel Libraries - Use MKL - Parallel
(The setup above enable the project to be linked to MKL's cblas routines.)



9. Removing cblas from GSL code.
- Exclude the cblas directory from the project
- Exclude the file gsl\gsl_cblas.h
- Remove the line
     #include <gsl/gsl_cblas.h>
  and replace if necessary with
     #include <mkl_cblas.h>
  for these files:
blas\blas.c
doc\examples\cblas.c    // probably not needed
eigen\francis.c
eigen\nonsymmv.c
gsl_blas_types.h

The rationale or excuse for not using GSL's cblas is that since we are building GSL with Intel C/C++ compilers, we should have license to the Intel MKL libraries too. Also Intel MKL should be highly optimised, thus hopefully more efficient.


10. Minimum  packages needed to compile DLL are:
block, complex, err, ieee-utils, matrix, sys, vector


11. Interoperability - Using DEF file method for exporting variables and functions.
- In the GSL project <proj>, right click and add new file.
- Choose the new file as a Module Definition file (*.def)
- In the contents of the file, type the following:
LIBRARY <proj>
EXPORTS
   funcA   @1
   funcB   @2
   funcC   @3
..... where funcA, funcB, funcC are the names of the functions to be exported. The ordinals @1, @2, @3 need to be consecutive numbers with no gaps in between. For functions exported in this manner, do not declare with __declspec(dllexport). It is possible to declare using both __declspec(dllexport)  and in DEF files, in this case, DATA or CONSTANT is used instead of ordinal numbers.
REF: MSDN Library - Importing and Exporting - Importing into an Application - Importing using DEF files.

- Ensure <proj> properties have the following Pre-Processors in addition to existing ones:
     WIN32, GSL_DLL, DLL_EXPORT
- go to gsl_ieee_utils.h and add the following near the top:
     #include <gsl/gsl_types.h>
- Ensure <proj> properties - Linker - Input - Module definition file : gslIntel.def
- Ensure the <consumer> project properties have the following Pre-Processors in addition to existing ones:
     WIN32, GSL_DLL    ....... do not have DLL_EXPORT
- Ensure the <consumer> project has these properties:
     Linker - Dependencies: gslIntel.dll
     Additional Library Directories: <location of gslIntel.lib>


12. Inline functions.
The following packages are affected: bspline, cdf
MSVC++ accepts both inline and _inline BUT when the code is *.c and compiled with MSVC, then it only recognize _inline.
Error Messages are like:
Error 1 error C2054: expected '(' to follow 'inline' ...\specfunc\cheb_eval_mode.c
Error 2 error C2085: 'cheb_eval_mode_e' : not in formal parameter list ...\specfunc\cheb_eval_mode.c
Error 3 error C2143: syntax error : missing ';' before '{' ...\specfunc\cheb_eval_mode.c 6
Solution:
In specfun/cheb_eval_mode.c, airy.c , gamma.c, bessel.c, bessel_olver, bessel_zero, coupling.c, ellint.c
erfc.c, legendre_con, trig.c, zeta.c
change from inline to _inline

***********************
  New INLINE Method
***********************
To AVOID changing any C code regarding "inline", creata a file "gsl/mygslIntel.h" and put this content into the file:
      #define inline _inline
Then put the following line
 #include <gsl/mygslIntel.h>
in the following files:
      config.h
 bspline/bspline.h
      gsl/gsl_sys.h
 gsl/gsl_sort.h
      gsl/gsl_heapsort.h
 gsl/gsl_rng.h
 interpolation/integ_eval.h

13. Excluded Source files.
As mention in a previous step, most *.c files within a package need to be "Include in Project". Then these types are generally excluded from BUILD:  *_inc.c, *_source.c and test*.c. However there are special cases where other *.c files also need to be EXCLUDED from BUILD - they are listed here.
cdf:      
eigen:       qrstep.c
fft:         exclude ALL files except - dft.c, fft.c, signals.c
histogram:   find.c, find2d.c, urand.c,
ieee-utils:  exclude ALL files except - endian.c, env.c, fp.c, make_rep.c, print.c, read.c, standardize.c
integration: append.c, cquad_const.c, err.c, initialise.c, positivity.c, ptsort.c, qpsrt.c, util.c,
             reset.c, qelg.c, qc25c.c, qc25f.c, qc25s.c, qpsrt2.c, set_initial.c,
linalg:      apply_givens.c, givens.c, svdstep.c
multifit:    lmiterate.c, lmpar.c, lmutil.c, lmset.c, qrsolv.c,
multimin:    directional_minimize.c, linear_minimize.c, linear_wrapper.c,
multiroots:  dogleg.c, enorm.c,
ode-initval2:control_utils.c, modnewton1.c, rksubs.c, step_utils.c
poly:        companion.c, balance.c, qr.c
rng:         schrage.c
specfun:     cheb_eval.c, cheb_eval_mode.c

To discover these *.c files to exclude, they are the ones usually included in other *.c.
So use a utility like BareGrep to scan all files for {#include "cheb_eval.c"} as an example. If that exist, then cheb_eval.c should be excluded from build compilation.


14. This section describes how to build the GSL-provided tests using VS2010 and the Intel C compiler.
The source file test.c is physically located at, eg:
      GSL_staging/gslIntelSoln/<proj>/sys/test.c
Creating Test projects for each package specifically:
- In the solution gslIntelSoln,  Add New Project - Visual C++ - Win32 - Win32 Console Application - "test_<gsl package>" -
  choose Console Application and Empty Project.
- Transform this to Intel C/C++ project by: right-click project - Intel C++ Composer XE 2011 - Use Intel C++
- In the test project, eg gslIntelSoln/test_block in Solution Explorer, right click on Source Files - Add Existing File.
- Choose the files to be tested by navigating to the gslIntelSoln/<proj>/block/ directory and select the test files such as test.c or others if applicable.
- Fill in for ALL CONFIGURATION, ALL PLATFORMS the following properties for the test project:
  C/C++ - General - Additional Include Directories - $(SolutionDir)gslIntel
  C/C++ - Preprocessor - WIN32, ...., GSL_DLL
  Linker - General - Additional Library Directories - $(OutputPath)
  Linker - Input - Additional Dependencies  - gslIntel.lib
- Compile and note the "error LNK2019" messages. If some variables or functons are not exposed, then edit the file
      gslIntel/gslIntel.def
  to add the functions and variables and recompile gslIntel and test projects.
- For the following test projects: test_eigen, test_linalg, test_wavelet, test_multifit, test_multimin,
  they use MKL and this can be configured by:
     - Project Propertie - Configuration Properties - Intel Performance Libraries - Intel Math Kernel Libraries - Use MKL - Parallel.
- For specfun test project, there is no test.c. Instead there are numerous test_sf* which must be included in test_specfun project.


15. Optimization
The following are various settings for the final version of the project, the gslIntel and all the test projects, hence usually apply to the Release configuration. These settings ensure the final product are not debug version, and also optimized by the compiler. In the project properties:

- under C/C++ - Preprocessor
    for test projects: WIN32, ....., _CONSOLE, GSL_DLL
    for gslIntel: WIN32, ....  GSL_DLL, DLL_EXPORT
ensure _DEBUG is not in any of them
- under C/C++ - Optimization
    Optimization: /O2, /Ox, etc..
    Enable Intrinsic Functions:   more testing needed about this option
- under C/C++ - Code Generation
Basic Runtime Check: Default
Runtime Library: Multi-threaded DLL  (Not Debug version)
- under C/C++ - Code Generation [Intel C++]
Intel Processor-Specific Optimization: more test needed
- under Linker - Debugging
      Generate Debug Info: No

Troubleshooting
=================
For crashes or compiling error first check the following:
- DO NOT / NEVER change any of the gsl source code such as *.c and *.h. In most cases Compile or Link failures is due to incorrect property settings in the VS solution gslIntelSoln or VS project settings gslIntel or other test_<package> project properties. There may be some different default settings in your VS compared to the VS settings used to build GSLWintel.
- Check preprocessor settings under Project Properties - C/C++ - Preprocessor
    for test projects: WIN32, ....., _CONSOLE, GSL_DLL
    for gslIntel: WIN32, ....  GSL_DLL, DLL_EXPORT
    The _CONSOLE ensures the test projects produce *.exe, while gslIntel must not have _CONSOLE because it needs to produce *.dll.
The DLL_EXPORT ensures the gsl variables in gslIntel.dll are visible to the outside world. The test projects are not export gsl variables hence must not have DLL_EXPORT.
- make sure MKL is used by specifying in:
    Project Properties - Configuration Properties - Intel Performance Libraries - Use MKL - PARALLEL option is on.


GSL-C-Fortran Framework
=========================
This section documents the projects and wrappers that need to be created to make use of gslIntel from Fortran projects.
The first part is to create a C-wrapper project (gslIntelCwrap) to create functions that will be called directly by other projects. These functions will make use of GSL functions internally. Hence gslIntelCwrap is a middle layer, used by other projects but internally calls GSL functions.
The second part is to write the appropriate interfaces for Fortran to call C functions.

1. At this stage there exist the GSL library with the following components:
gsl/gsl_xxxx.h      the header files
gslIntel.lib        - compile time library
gslIntel.dll        - runtime library

2. In VS2010, create a wrapper C++ project, say gslIntelCwrap. Create this as a C++ Win32 application, DLL, empty project (do not choose Export Symbols option).
3. Create a header file gslIntelCwrap.h and code file gslIntelCwrap.c
4. In the header file gslCwrap.h
a) prepare to export the wrappers.

extern "C"{
    __declspec(dllexport) void  mySinh(double x, double *result);
}

b) include reference to GSL header like and call gsl function:
#include "gslIntelCwrap.h"
#include "gsl/gsl_sys.h"
void mySinh (double x, double *result) {    
*result =  gsl_asinh(x);           // the actual gsl function
}// 2 * x;  }

5. Go to the gslIntelCwrap project properties page and fill the following:
- C/C++ - General - Additional Include Directories - <directory of gsl\gsl_xxx.h>
- Linker - General - Additional Library Directories - <location of gslIntel.lib>
- Linker - Input - Additional Dependencies - gslIntel.lib


6. To call from Fortran,
i) prepare a Fortran interface:
    interface
    SUBROUTINE fmySinh(x, dresult)  BIND(C, NAME='mySinh')
        USE, INTRINSIC :: ISO_C_BINDING     ! New Fortran 2003 standard
        IMPLICIT NONE
        REAL(C_DOUBLE), VALUE :: x
        REAL(C_DOUBLE) :: dresult
    END SUBROUTINE fmySinh
end interface

ii) In the Fortran code, just call the function as specified by the interface.
Eg.
    call fmySinh(1.2d0, answw)

iii) Add the following project properties before compilation.
- Linker - General - Additional Library Directories - <location of gslIntelCwrap.lib>
- Linker - Input - Additional Dependencies - gslIntelCwrap.lib   OR
  put the following line somewhere in the Fortran code:
             !dec$objcomment lib:'gslIntelCwrap.lib'

iv) Ensure the following dlls are in the path seen by the Fortran executable:
gslIntel.dll, gslIntelCwrap.dll


7. There are at least 3 projects involved here.
gslIntel - this keeps the gsl code in its own project, so that it is independent of any change in interface or usage.
gslIntelCwrap - this consist of the C wrappers so that it can be used by C code or Fortran code. It also transforms all gsl functions into VOID C functions so that it can be called as Fortran subroutines. Fortran Interface - this is freely linked to the gslIntelCwrap wrapper code. There is no direct reference to the GSL code.  

8. error LNK2019: unresolved external symbol
This error can occur with some of GSL functions which have not been exported.

Friday, July 13, 2012

Notes WinDBG


This article shows how to GET and INSTALL and use WINDBG.

Goal: To analyse *.dmp file from Blue Screen of Death (BSOD)

OS: Windows 7 SP1 or Windows Server 2008 R2

Ref: http://windows7themes.net/how-to-open-dmp-files-in-windows-7.html


1. To get the WinDBG.exe, go to:
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=8279
This is the site called Microsoft Windows SDK for Windows 7 and .NET Framework 4

Discovering this site was a challenge. I first type "windbg" on Google. Then the usual Microsoft run around actually pointed to the Windows 8 RC version for this tool. But the link above is the correct one for Win7 and Win 2008 ServerR2.

Although we need WinDBG only, it comes as part of the Windows Driver Kit or the SDK as well.
The site above lets you download "winsdk_web.exe" 498K. This is a web setup - which is why the size is so small.



2. Install WinDBG.
Click on winsdk_web.exe
You may get an error if you choose to install everything.
I needed WinDBG only so I selected the following to install:
- .Net Development
    Intellisense and Reference Assemblies
    Tools
- Common Utilities
    (ALL)
- Redistributatble Packages
    Debugging Tools

3. Run WinDBG.exe
When installed properly, it is generally located in:
C:\Program Files\Microsoft SDKs\Windows\v7.1\Redist\Debugging Tools for Windows
.... or something like that.

From the Start Menu, type WinDBG on the search bar.

Or from the Start Menu - All Programs - Debugging Tools for Windows - WinDBG


4. Get Debugging Symbols
This is a one time setup.
- Create a new folder called C:\Symbols
- Open WinDBG
- Click on File – Symbol File Path
- Insert the following path:
SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols


5. Analyse DMP files.
Usually BSOD dump files are located in: C:\Windows\Minidump....
In WinDBG, click File - Open Crash Dump, then select your dump file.

Some information is displayed on the Window. If you need more information, click on the text called "!analyze -v" in the WinDBG window.

Friday, June 15, 2012

How To Install Windows Server 2008 R2 Enterprise Edition

Here is a list of steps used to install Windows Server 2008 R2 Enterprise Edition.

Purpose of writing this down is for me to have a record, and also to help others who may come across difficulties in doing the same thing.

Preamble: There is an infinite number of ways to install WinServer 2008. The way presented here fulfills my requirement that the final system is:
1. Dual boot with Windows 7 in Disk 1 and WinServer 2008 in Disk 2.
Basically that's it.

Disk Preparation:
To prepare the new disk by partitioning, an excellent tool is the free MiniTool Partition Wizard (http://www.partitionwizard.com/free-partition-manager.html).
This is a Linux bootable DVD and allows partitions to be manipulated, including primary and logical. Format can also be done in NTFS or ext2/ext3 and maybe more.
So partition each drive before installing the Operating System (OS).

Dual Booting Requirements:
The Boot Manager used here is Boot-US (www.boot-us.com). It recommends to install Windows 7 with Disk 1 connected. Then remove Disk 1 and attach Disk 2. Install WinServer 2008 on Disk 2. Then attach Disk 1, boot to Windows 7 (no choice yet). Install Boot US and configure Boot Manager. The detailed instructions is found in Boot-US tips here:

"Installing Windows multiple times on different disks" http://www.boot-us.com/tips_w02.htm

As it turns out after many panic attacks, the way I have gotten it to work is:
Disk 1: Win7 and Win Server 2008R2.
Disk 2: Data drive

The trick to install both OS is:
- install first OS
- switch the second partition as ACTIVE. This can be done using the BootUS installed in the first OS.
- reboot and install second OS.
- to switch between the 2 partitions, change the Active partition switch.

Configuring Wireless Network:
This is one of the trickiest steps in WinServer 2008 installation. A very good site is:
Enabling Wireless Network in Windows Server 2008,
(http://geekswithblogs.net/NewThingsILearned/archive/2009/01/30/enabling-wireless-network-in-windows-server-2008.aspx). Basically the steps are:
1. After WinServer 2008 installation has finished, there will be a list of items to configure for.
2. Choose "Add Features", then choose "Wireless LAN Service".
3. After adding this feature, a list of neighboring Wireless networks can be seen in by doing this: click on the "Network and Sharing Center" and click "Connect to Network".
4. In that same "Network and Sharing Center", click on "Manage Wireless Network", click on "Add", then add your wireless details.
5. When filling in the network details, do not check the option for "Connect even if network is not broadcasting its name (SSID)". There is also a tip to suggest not to check this option.
6. Although now the wireless is setup, going to "Connect to Network" show other wireless network but not our own. Go in and out of this dialog for a while, then come back and look at the list of wireless networks. At the end of the list, click on the network called "Others", this will ask for your hidden SSID and password. After this step, the network should be connected. Check this by using a web browser.

Security:
Firewall - Win Server 2008R2 comes with an advanced firewall, and hence does not need 3rd party firewall. More details on this is found in :
http://www.windowsnetworking.com/articles_tutorials/configure-Windows-Server-2008-advanced-firewall-MMC-snap-in.html

AntiVirus - While 3rd party antivirus exist for Win Server 2008 R2, they usually cost much higher than their normal PC counterparts. Even a few of the popular free antivirus have Server versions, but they are Not Free. One practical solution is to use Microsoft's own Security Essentials, coupled with various small anti spyware programs.

Thursday, May 17, 2012

Notes SVN


NotesSVN

Summary
Repository creation
Start the server
Start the server as a deamon
Installing SubEclipse plugin: Subversion plugin for Eclispe
Check-in project into SVN for first time, from Eclipse
Delete Project and Checkout Project from Eclipse
Other Help
Command Line
Windows - CollabnetSubversion - Upload Google Code
Visual Studio and AnkhSVN
SVN Concept
SVN status Flags
SVN server configuration workflow


Summary
========
Reference:http://svnbook.red-bean.com/en/1.1/svn-book.pdf

Subversion is installed on Fedora distributions:

svn             The command-line client program.
svnversion      reporting the state (in terms of revisions of the items present) of a
                working copy.
svnlook         inspecting a Subversion repository.
svnadmin        creating, tweaking or repairing a Subversion repository.
svndumpfilter   filtering Subversion repository dump streams.
mod_dav_svn     plug-in module for the Apache HTTP Server, used to make your repository
                available to others over a network.
svnserve        custom standalone server program, runnable as a daemon process or invokable by SSH;
                another way to make your repository available to others over a network.

Repository creation
===================

User "svn" created (passwd svn), mmsn's group member (chee, quilici als belong to this group). This account
will host the repository.

Log as svn user.

svnadmin create --fs-type fsfs /home/svn/mmsn
This creates the repository mmsn, that will contain all projects. Note that --fs-type fsfs force s to use fsfs instead of Berkley DB

Importing initial data
======================
0. Login as svn user

1. Create SVN style directory structure, on local machine, for the code to be imported.
$ mkdir tmpdir
$ cd tmpdir
$ mkdir BIS
$ mkdir BIS/trunk
$ mkdir BIS/branches
$ mkdir BIS/tags
$ cd BIS/trunk
$ mkdir bin build config lib src ....
$ cd ../..     i.e. go back up to ..../tmpdir  from ..../tmpdir/BIS/trunk

2. Import files manually or by eclipse
svn import . file:///home/svn/mmsn --message 'Initial import'
Adding         BIS
Adding         BIS/trunk
Adding         BIS/trunk/build
Adding         BIS/trunk/build/build
Adding         BIS/trunk/build/build.bat
Adding  (bin)  BIS/trunk/build/xml-apis.jar
Adding  (bin)  BIS/trunk/build/xercesImpl.jar
Adding         BIS/trunk/build/build.properties
Adding  (bin)  BIS/trunk/build/ant-launcher.jar
Adding  (bin)  BIS/trunk/build/ant.jar
Adding         BIS/trunk/build/build.xml
Adding         BIS/trunk/config
Adding         BIS/trunk/config/log4j.properties
Adding         BIS/trunk/config/bisJalopy.xml
Adding         BIS/trunk/lib
Adding         BIS/trunk/lib/licenses
Adding         BIS/trunk/lib/licenses/log4j.LICENSE
Adding  (bin)  BIS/trunk/lib/log4j-1.2.8.jar
Adding         BIS/trunk/src
Adding         BIS/trunk/bin
Adding         BIS/branches
Adding         BIS/tags


Committed revision 1.

$ cd ..
$ rm -rf tmpdir

$ cd mmsn
$  chmod -R 775 . This gives mmsn's group members r+w+x permissions in the repository

Following is not needed, just kept to remember
=====================================================================
#Start the server
#================
#svnserve -t -r /home/svn/. -t option means that we are going to use ssh to connect svn server, -r should allow to use relative paths
#when invoking the server (-r does not seem to work with -t option !!!!)
#
#Start the server as a deamon
#============================
#create a file under /etc/init.d: svnd
#
# more /etc/init.d/svnd
#!/bin/bash
#
# gdm-bootup-complete     This shell script removes the boot throbber from
#                         the login screen
#
# chkconfig:    5 99 1
# description: This shell script removes the boot throbber from the login screen

# source function library
#. /etc/rc.d/init.d/functions
#
#start() {
#        daemon --user=svn svnserve -t  1> /home/svn/svnd.log 2>&1
#}
#
#stop() {
#        killall -KILL svnserve
#}
#
#case "$1" in
#  start)
#        start
#        ;;
#  stop)
#        stop
#        ;;
#  *)
#         echo $"Usage: $0 {start|stop}"
#        exit 1
# esac
#
# exit 0
#
#Then add the svnd service with chkconfig:
#/sbin/chkconfig --add svnd
#/sbin/chkconfig --level 5 svnd on
#
#
#Change Security Context of some libraries (only on Linux running Fedora or Red Hat with SecureLinux turned on)
#chcon -t texrel_shlib_t /usr/local/lib/libsvn_fs_fs-1.so.0
#chcon -t texrel_shlib_t /usr/local/lib/libsvn_ra_svn-1.so.0
#chcon -t texrel_shlib_t /usr/local/lib/libsvn_repos-1.so.0
#chcon -t texrel_shlib_t /usr/local/lib/libsvn_subr-1.so.0

=====================================================================

Installing SubEclipse plugin: Subversion plugin for Eclispe
============================================================
0. If using a proxy, fill in proxy details at:
    Windows -> Preference -> Install/Update
1. Install plugin from Eclipse:
 a) Help -> Software Updates -> Find and Install -> Search for New Features to Install ->
 New Remote Site -> Fill in URL="http://subclipse.tigris.org/update_1.0.x"
 b) Click finish and follow update instructions.
 c) for deetailed instructions: see  http://subclipse.tigris.org/install.html
2. Change configuration in Eclipse:
   Windows --> Preferences --> Team --> SVN panel, change the SVN interface to JavaSVN(Pure Java)
3. Connect using ssh
   a) Window -> Show View -> Other -> SVN -> SVN Repository
   b) Right click any space within SVN Repository. New -> Repository Location:
   c) Enter: svn+ssh://enrol1/home/svn/mmsn   (repository that is created on the SVN server already)
   d) Choose authenticate by password or Key. If choose Key option, enter the location of the private-key file of the local machine. Ensure public key of local machine has been added to authorized_keys file of the remote machine.
4. SVN through http proxy
   a) Goto where subversion is installed, eg:
      c:\Documents and Settings\<user>\ApplicationData\Subversion\
   b) Edit the file called "servers" and modify the variables fors:
      http-proxy-host
      http-proxy-port
 

Check-in project into SVN for first time, from Eclipse
=======================================================
1. In Eclipse, right click on the project to be deposited into SVN for first time.
2. From the Context menu, select Team -> Share Project


Delete Project and Checkout Project from Eclipse
=================================================
1. Go to Eclipse and click on selected Project (eg. Notes[Notes\trunk]) and close the project
2. Delete contents of the project (Make sure you have a backup of all the files somewhere else)
3. From Eclipse, New->Other->Checkout project from SVN, using location:
      svn+ssh://<host>/home/svn/mmsn
   ... where host is the hostname (e.g. arnold)
4. Select Notes/trunk
5. Checkout using Project Wizard.



Other Help
==========
Check the manual in the Elipse's help section to get started.
Check the help manual for the commands.



Command lines
=============
To checkout the code
svn co svn+ssh://arnold/home/svn/mmsn/BIS/trunk/ BIS
To update
svn update --> check options in the manual

To rename, even when filenames have spaces
svn move svn://localhost/home/svn/QuantAnalytics/second\ text\ file  svn://localhost/home/svn/QuantAnalytics/Seee
svn update



Windows - CollabnetSubversion -  Upload Google Code
====================================================
Create local repository
- svnadmin create c:\svn_repo

Upload first item into local repository
- svn import u:\Notes\file.txt file:////c:/svn_repo/Notes -m "initial creation"

For sync'ing with Google the First Time, click on "reset this repository" in google code -> Source -> Checkout.

Syncing local repository to Google Code repository
 - svnsync init --username <Google User name> https://<webproject>.googlecode.com/svn file:////c:/svn_repo
 >>> Copied properties for revision 0.
 - svnsync sync --username <Google User name> https://<webproject>.googlecode.com/svn
 >>> Transmitting file data .
 >>> Committed revision 1.
 >>> Copied properties for revision 1.



 Visual Studio and AnkhSVN
 ===========================
 Assuming the repository has been created.
 1. Create a new folder in the repository.
    From VS.Net open the Repository Explorer by right click.
    Create a folder called myWeb say, in the repository.
2. To map h:\myWeb, where myWeb already has contents:
   Close any solutions.
   Create new Project -> Other Project Types -> Visual Studio Solutions -> Blank Solutions.
   Name = myWeb
   Location = H:\
   Add to Subversion
3. Fill in the Subversion path.
4. Start adding files.
   Click on the generic "Solution" in Solution Explorer and "Add Existing Items"  


SVN Concept
=============

Repository Vs Working Copy
- files that are edited are Working Copy Files.
- Working Copy is created by "checkout" the files from the Repository

File Status:  Unchanged Vs Locally Changed
- "svn commit" will do nothing if the file is unchanged.
- If Locally Changed, "svn commit" will commit changes to repository unless the file is Out of Date.

File Status:  Current Vs Out Of Date
- If Repository file has changed since last checkout or update, then it is Out of Date.
- "svn update" will merge local file into repository file.

svn checkout <SVN_path>/<Project Dir>
- Warning, the project path including the project name will be copied into the current directory. So ensure the current directory is the Parent of the intended project directory.

svn commit
- Push files to repository

svn update
- Pull filed from repository



SVN status Flags
================
A   The file, directory, or symbolic link item has been scheduled for addition into the repository.
C   The file item is in a state of conflict. That is, changes received from the server during an update overlap with local changes that you have in your working copy (and weren't resolved during the update). You must resolve this conflict before committing your changes to the repository.
D   The file, directory, or symbolic link item has been scheduled for deletion from the repository.
M   The contents of the file item have been modified. If you pass a specific path to svn status, you get information about that item alone:



SVN server configuration workflow
===================================
RHEL5 has got subversin 1.6.11 pre-installed for both i386 and x86_64
To use Subversion with R, the work needs to be organized a Rstudio Projects.

1. To check if the svnserver starts automatically, whether svn is preinstalled for Red Hat, or self install later, type:
  ls -laF /etc/init.d/svnserve
to see if the file exists. This file runs a script to start the daemon.

2. Create svn user and group, type:
sudo useradd svn  - use id svn to check user details
sudo passwd svn   - make password for svn -> "svn"
sudo usermod  -a -G R_POC_TEAM svn    - this puts the user "svn" to the group called R_POC_TEAM.

Ensure that all other user of svn is in the same group called "R_POC_TEAM". To check which groups the users are in, type:
sudo more /etc/passwd

3. Create the SVN repository for all members of group.
- this is done only ONCE, when a new VM and subversion is installed.
- all users and the 'svn' user need to be in the same group. In this example, all these users belong to a group called R_POC_TEAM. If this is not true, make it so.
- Login as svn user.
- create the repository called QA, type:
svnadmin create --fs-type fsfs /home/svn/QA

This creates the repository QA, that will contain all projects. Note that --fs-type fsfs is the preferred filesystem for svn rather than the Berkley DB filesystem (see manual).

4. Check the file /etc/services. If it does not contain the following lines, then add them:
svn    3690/tcp     # Subversion
svn    3690/udp     # Subversion

5. Configuring Authentication and Authorization
- LOGIN AS USER called 'svn'
- go to the QA repository, eg:
    cd /home/svn/QA
- Create a file under ...QA/conf/svnserve.conf, with the following content:
[general]
password=db = passwd
realm = QA realm
anon-access = read
auth-access = write
...... passwd is actually the word "passwd".

- Ensure the file called 'passwd' exist under ...QA/conf/ and that it has the following content:
[users]
harry = harryssecret
sally = sallyssecret

.... where harry and sally are valid users in the linux box.


6. Hide the absolute path of the svn server and provide relative paths to user only.
Edit this file: /etc/rc.d/init.d/svnserve

by adding "-r /home/svn" to the following line if it exist, so that it becomes:
args="--daemon --pid-file=${pidfile} $OPTIONS  -r /home/svn"

Now when the svn server is mentioned, it is done by:
<svn server>/QA   instead of  <svn server>/home/svn/QA


The following information are for references:
- Access SVN. The URL pointing to the repository is:
svn://<HOST>/QA
svn://localhost/QA

- Some svn admin commands: svnadmin, svnlook, svndump, svndumpfilter, svnsync

- svnlook info <repos>      - prints information for the repository at <repos>.
                            The information include author, date, number of lines of log, log message

- sudo killall svnserve     - force termination of any active svnserve services. Also solves the error with message:
                            " svnserve: Can't bind server socket: Address already in use"

- sudo /etc/init.d/svnserve start       - to start svn server once off.

***** How to setup iptables - NOT Needed here
http://articles.slicehost.com/2007/9/5/introduction-to-svnserve

Notes Linux



Contents

- 32bit vs 64bit
- Bitmap Fonts
- CDwriting
- WinXP and Lilo booting sequence
- SSH Tunnelling Enabling and VNC
- SSH X11 port forwarding
- USB Mounting and /etc/fstab editing
- Loading Profiles
- Linux Firewall
- Linux Runlevels
- Block web-ads
- SELinux Configuration
- For using SELinux with cgi-bin and files in Symbolic links
- RPM
- Installing Source RPMs
- Linux Commands

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

32bit vs 64bit
==============

CPU 32 bit -> integer pointers are 32bits
2 to power 32 = 4e+09   about 4 Giga stuff

Real*8 is 8 bytes - i.e. 64 bits but these are a separate thing
altogether from integer pointers.

Pointers (a C/C++ concept - alien to Fortran)
Consider an array of REAL*8 Arr(17:27)
   there are 11 elements, i.e. from Arr(17), ...., Arr(27)
   each element can be referenced with a pointer so we need 11 pointers.
   but if the pointer size is 2^32 (4e+09),
      so the max array size(w.r.t. #elements) possible is 4e09





Bitmap Fonts
============
Eg. Installing Korean Fonts

0. Log in as root

1. Download Korean Fonts and put into /scratch/chee/fonts/Korean-X
The files are:
hanglg16.pcf.gz  hanglm16.pcf.gz  hanglm24.pcf.gz

If the files are in bdf format then,
i) convert to pcf:
   bdftopcf <file>.bdf
ii) compress it
   gzip <file>.pcf
or
   compress <file>.pcf

2. Create an index file for the fonts in that directory
   mkfontdir /scratch/chee/fonts/Korean-X

this will create fonts.dir in the Korean-X directory

3. Tell the Xserver where to find the new fonts.

   xset fp+ /scratch/chee/fonts/Korean-X
   or
   xset +fp /scratch/chee/fonts/Korean-X

to check the current X settings
   xset q

to add this permanently, add the xset fp to .xinitrc

4. Refresh the Xserver setting when new fonts added if necessary

xset fp rehash

5. To view fonts in IRIX

xfd

CDwriting
=========
using cdrtools (cdrecord, mkisofs, etc..)
Ref: http://wt.xpilot.org/publications/linux/howtos/cd-writing/html/

1. Collect files and make a iso filesystem
mkisofs  -r   -o cd_image   private_collection/

where
-r   = Rockridge extns (deeper than 8 levels, permissions, etc)
-J   = Joliet extensions - more MS Windows friendly
-o   = output filename
-v   = verbose
-x   = exclude directory path
for second session only:
-C x,y = x is last session start, y is next session start
       = use "cdrecord msinfo" to get -C x,y info
       = must use with -M option
-M path = path where multi session is to merge. Usually path is
        = place normally mount CDROM eg /mnt/cdrom
EG:
cd /scratch/chee/software
mkisofs -v  -r -J -o firstImage.raw  forWinPC/AdbeRdr601_enu_full.exe

*****
mkisofs 1.12.1 (i686-pc-linux-gnu)
 61.14% done, estimate finish Fri Sep 10 16:32:27 2004
Total extents actually written = 8189
Total translation table size: 0
Total rockridge attributes bytes: 264
Total directory bytes: 0
Path table size(bytes): 10
Max brk space used 5544
8189 extents written (15 Mb)
*****


2. Test Image
Mount the iso image (cd_image) into the directory /cdrom, so you can check the directories.

mount -t iso9660 -o ro,loop=/dev/loop0 cd_image /cdrom

3. Writing to CD
to test the CD writer and get info

cdrecord -scanbus

to write to the CD
shell> SCSI_BUS=0   # taken from listing 1 "scsibus0:"
shell> SCSI_ID=6    # taken from listing 1 "TOSHIBA XM-3401"
shell> SCSI_LUN=0
shell> cdrecord -v speed=2 dev=$SCSI_BUS,$SCSI_ID,$SCSI_LUN \
                   -data  cd_image

# same as above, but shorter:
shell> cdrecord -v speed=2 dev=0,6,0 -multi -data  cd_image

where
-dummy = simulation (laser off)
-eject = eject after burning
-v    = verbose
-multi = multi-session
-msinfo = get multi-session info
-toc = get table of contents
-data = data file (mode1 YellowBook)
-audio = audio files
speed=x  = x is speed for writing
dev = 0,0,0 for parallel.hpc.unsw.edu.au

EG
cdrecord -dummy -v speed=2 dev=0,0,0 -multi -data  firstImage.raw


4. For CDRW
writing the 2nd, 3rd sessions onwards

shell> NEXT_TRACK=`cdrecord -msinfo dev=0,6,0`
shell> echo $NEXT_TRACK
shell> mkisofs -R -o cd_image2 -C $NEXT_TRACK -M /dev/scd5 private_collection/ \

EG.
cdrecord -msinfo dev=0,0,0
   0,19591

mkisofs -v  -r -J -o secondImage -C 0,19591 -M /dev/cdrom forWinPC/Firewall

cdrecord -dummy -v speed=2 dev=0,0,0 -multi -data  secondImage


WinXP and Lilo booting sequence
===============================
   - Example for Slackware Linux

   - use the linux diskette to startup linux

   - login as root

   - modify the /etc/lilo.conf file as follow:

     lba32
     boot = /dev/hda1
     vga = 773
     image = /boot/vmlinuz
     root = /dev/hda1
     label = Slackware                <-- no space
     read-only

   - then do:

     $ lilo -b /dev/hda1              <-- lilo on hda1
     $ dd if=/dev/hda1 of=/dos/boot.lnx bs=512 count=1
     $ shutdown -r now

   - take out the linux boot diskette and re-boot the
     system to WinXP, then do:

     ** copy the file from d:\dos\boot.lnx to c:\
     ** then, modify the file c:\boot.ini to include
        following lines:

        [boot loader]
        timeout=30

        [operating systems]
        c:\boot.lnx = "Slareware Linux 9.1"
        ...

   - re-start the Wxp and there should be an option on the Wxp
     loader to boot to Linux partition.

   - If you only want to boot to the Linux partition only, just set
     the active partition to the Linux installation partition (in this
     case, it is hda1) and reboot the Windows again.


SSH Tunnelling and VNC
======================

   ssh -L x:localhost:y anotherhost

means connect to anotherhost using ssh and listen to port x on the local machine and forward any connections there to port y on anotherhost.

To start a VNC server type:

   vncserver

To start a VNC viewer type:

   vncviewer <name.of.remote.machine>:<display.number>

Using secure VNC with ssh, if a VNC server is running as display:1 on a system called linuxserver, you can start a secure SSH session by typing:

   ssh -L 5902:localhost:5901 linuxserver

So now, connecting to port 5902 on a local machine actually connects to port 5901 on linuxserver. Thus to connect from a client machine:

   vncviewer localhost:2


SSH X11 port forwarding
========================

On SSH server (remote host):
To check SSH is setup for X11 port forwarding, look at the file:
/etc/ssh/sshd_config
and see if the statement is present:
X11 Forwarding yes
then restart X11
/etc/init.d/sshd restart

On Cygwin (client):
- copy /etc/defaults/etc/ssh_config into ~/.ssh
- ensure the following are set:
ForwardX11 yes
ForwardX11Trusted yes
- DO NOT export DISPLAY=localhost:0.0

To ssh with X11 forwarding (may be different for different Unix/Linux)
ssh -X <name>@<host>



USB Mounting
=============
1. must be root to do this.
 mount -t vfat /dev/sda1 /mnt/usb/

2. Alternatively, modify /etc/fstab file to have this line
/dev/sda1        /usbdisk         vfat        user,exec            1   1

Then any user can type
  mount /usbdisk

and the files on the usb will have User's permissions

3. To unmount, type:
   umount /mnt/usb/
       or
   umounut /usbdisk


Loading Profiles
================
/etc/profile  ->  /home/<user>/.profile  ->  /home/<user>/.bashrc

Master startup scripts are in /etc :
bashrc profile csh.cshrc csh.login

/etc/profile.d   contains application specific Macro variables, eg.
java.sh java.csh



Linux Firewall
===============
/sbin/iptables --- executable
/etc/init.d/iptables                            --- script file
/etc/sysconfig/iptables                         --- configuration file

To stop firewall
/etc/init.d/iptables stop

To start firewall
/etc/init.d/iptables start

To check firewall
/sbin/iptables -L --line-numbers                --- to see various rules
/sbin/iptables -nL <INPUT/OUTPUT/RH-Firewall>   --- to see various rules and ports

Chains
/sbin/iptables -F [chain]                       --- deleting all rules in the chain
/sbin/iptables -X [chain]                       --- delete optional user-def chain
/sbin/iptables -Z [chain]                       --- to reset / zero packet and byte counter
/sbin/iptables -N [chain]                       --- to create new user-def chain

<Target>: {Accept, Queue, Drop, Return}

Rule-specifications
/sbin/iptables -D chain rule                    --- delete rule in chain
/sbin/iptables -A chain rule                    --- append rule to end of chain
/sbin/iptables -I chain [rulenum] rule          --- insert rule at rulenum or 1st position of chain
/sbin/iptables -R chain rulenum rule            --- replace rule to at rulenum positin of chain

Rules specs: -<s/d> [!] IPaddr[/mask] [!] [port[:port]]
   !=negate, []=optional

Other Parameters
-i           --- interface, eg eth0
-p           --- protocol, eg tcp, udp, icmp, all
-s           --- source (see above)
-d           --- destination (see above)
-j           --- jump <target>
-m state     --- match with the state module
--line-numbers

<James Sainsbury help>
Example - Webserver httpd - open at port 80
iptables -A INPUT -p tcp -i eth0 --dport 80 -j ACCEPT - Open HTTP port

  or

So locate the rule in RH-Firewall-1-INPUT for WWW
eg    iptables --line -L RH-Firewall-1-INPUT

eg    9  ACCEPT  tcp  --  anywhere  anywhere  state NEW tcp dpt:http
and insert a rule just after that rule (9 here) so insert before rule 10

# iptables -I RH-Firewall-1-INPUT 10 -p tcp -s 0/0 -d enrol1 -dport 8080 -m state --state NEW -j ACCEPT
If you are satisfied the rule is working save the ruleset with
    /etc/init.d/iptables save
If you trash the ruleset but haven't saved just
    /etc/init.d/iptables restart
(You will notice there is a preceding rule
    7  ACCEPT  all  --  anywhere  anywhere  state  RELATED,ESTABLISHED
which accepts incoming packets as part of an ongoing connection.)
</James Sainsbury help>


To Save iptables:
/sbin/service iptables save



My Example for opening ports on local enrol1 to communicate with BIS on eb
iptables -I RH-Firewall-1-INPUT 12 -p tcp -s eb -d enrol1 --sport 49153:49155 -m state --state NEW -j ACCEPT


- To redirect all external traffic coming into port 80 into port 8080:
1. Use firewall configuration software to accept all TCP traffic from destination port 8080
2. (as root): iptables -t nat -A PREROUTING -p tcp --dport 80 -i eth0 -j REDIRECT --to-port 8080

- To redirect internal traffic coming into port 80 to port 8080
1. use program called rinetd.
2. in the /etc/rinetd.conf, have the line
"xx.xx.xx.xx 80 xx.xx.xx.xx 8080"
3. start the rinetd service



Linux Runlevels - Auto Startup
==============================
/usr/sbin/xxxd              --- daemon or startup file to run
/etc/xxx.conf               --- configuration file of the progran
/var/run/xxxd.pid           --- process ID of the file that run

Runlevels
0   - system shutdown
1   - single user, no networking or daemons, for maintenance
2   - multiuser but no NFS, can access network but others cannot get in
3   - normal multiuser, no X login
4   - undefined
5   - multiuser X Display manager, X login from consoles
6   - reboot
s/S - single user mode

/etc/inittab            - shows how the system is initialized
/etc/rc.d/init.d/         - list of daemon files, used or not used in startup
/etc/rc.d/rc3.d/        - links to actual files in init.d, at this particular runlevel 3


Program Sequence       Config file
at Startup
===========================================
/sbin/init             /etc/inittab

/etc/rc.d/init.d/*     /etc/sysconfig/daemons

/etc/rc.d/rcN.d/*
(Sym links of init.d)


Note the files in rcN.d are preceded by K or S and a number
K = kill job
S = start job
number = order of pograms

To check what services are running:
/sbin/chkconfig --list
    service <service name> status

- Remember to check Linux Firewalls if network services do not start

- edit script files in /etc/init.d/ to contain
1. start/stop functions (and others functions restart)
2. commands in start function MUST NOT BE EXECUTABLE commands. They
   should be executable scripts.
3. use daemon function to run as non-root user
4. DO NOT USE & (forking) at the end of command
5. Example:
        daemon --user=<userName> <ShellScriptName> 1> job.log 2>&1

- To register
/sbin/chkconfig --level 5 initdScript on










Block web-ads
==============

Most of us can't stand getting pummeled by ads everytime we go to a website. There
are a couple of ways to stop these ads.  One is to edit your /etc/hosts file and list
 your loopback address, 127.0.0.1, with the ad hosting address. For example, to
block ads from ar.atwola.com open up /etc/hosts as root:

[root@blueox]# vim /etc/hosts

Next, add the following line:

127.0.0.1 ar.atwola.com

This will line will tell your browser to look on 127.0.0.1 for the ads from
ar.atwola.com.  Since localhost doesn't have the ad, it won't be displated.
You can download a hosts file that has a multitude of ads to block.
Personally I DO NOT RECOMMEND THIS approach.  I have noticed that
my computer slows down trying to search through the hosts file.

The RECOMMENDED approach for firefox and mozilla users is to go to
the chrome directory of your personal browser folder.

[lumbrjackedpcj@blueox]# cd ~/.mozilla/firefox/dxshwykf.default/chrome

The directory dxshwykf.default will be different for you installation.

1. First, create the file userContent.css

[lumbrjackedpcj@blueox]# touch userContent.css

2. Next, edit this file with your favorite text editor

[lumbrjackedpcj@blueox]# vim userContent.css

3. Now, following the instrunctions on http://www.mozilla.org/support/firefox/adblock,
  copy this userContent.css and paste it in yours.



SELinux Configuration
======================
Intro
The latest releases of Linux distributions incorporate the mandatory access
control architecture SELinux of the NSA into the Linux kernels.
More information on SELinux can be found at the following link:
http://www.nsa.gov/selinux/info/faq.cfm

/usr/sbin/sestatus -v                      --- to get status of selinux

En/Disabling SELinux:
1 - in your /etc/sysconfig/selinux file, set:
SELINUX=<disabled|enforcing|permissive>
... this is not TRUE disable, it only switches off enforcing and skips loading policy.

2 - if using GRUB as bootloader
go to /boot/grub/grub.conf or menu.lst (symlink to grub.conf)
add this line to the correct boot section
selinux=<0|1>

3 - if using LILO (... to be completed ..., contributions welcome)


Error Message:
"error while loading shared libraries: /usr/lib/liba52.so.0: cannot restore segment prot after reloc: Permission denied"
then your SELinux configuration is preventing your application from launching.


Solutions:
For shared libraries failing to load, try:
chcon -t texrel_shlib_t /usr/lib/liba52.so.0
For some other libraries
chcon -c -v -R -u system_u -r object_r -t lib_t $InstallationDirectory/SystemFiles/Libraries/


# View file/directory contexts: ls -Z /var/www
# Set file/directory context:
    chcon -R -h -t httpd_sys_content_t /home/user1/public_html
Options:
    * -R: Recursive. Files and directories in current directory and all subdirectories.
    * -h: Affect symbolic links.
    * -t: Specify type of security context.
# View security context of running processes: ps -eZ
# View security context of a user: id
    uid=500(user1) gid=500(user1) groups=500(user1) context=user_u:system_r:unconfined_t
# Security contexts for a new file default to that of the parent directory.


For using SELinux with cgi-bin and files in Symbolic links
============================================================
Example:
perl script:      /var/www/cgi-bin/overview
Data file:        /home/BIS/BIS/log/ANGLE.txt
Symbolic Link:    /var/www/html/BIS/ANGLE.txt

The security context, especially the subdirectories that contain it are
as follows:

/var/www/cgi-bin/overview      root:object_r:httpd_sys_script_exec_t

/var/www/html/BIS              user_u:object_r:httpd_sys_content_t
/var/www/html/BIS/ANGLE.txt    user_u:object_r:httpd_sys_content_t

/home                          system_u:object_r:home_root_t
/home/BIS                      user_u:object_r:user_home_dir_t
/home/BIS/BIS                  user_u:object_r:httpd_sys_content_t
/home/BIS/BIS/log              user_u:object_r:httpd_sys_content_t
/home/BIS/BIS/log/ANGLE.txt    user_u:object_r:httpd_sys_content_t

Note that it is enough to ensure the "type" context at the /home/BIS/BIS
level and its contents seem to inherit the type.


RPM
===
Redhat package manager

Query
rpm -q -a                        --- list all packages
rpm -q -p <package_path.rpm>     --- list packages inside package_path rpm file
rpm -q -l -p <package_path.rpm>  --- list all files inside package_path rpm file
rpm -q -a -i                     --- list info on all packages
rpm -q -a -f                     --- list files of all packages



Installing Source RPMs
=======================
1. Get source code in rpm form from the "SRPMS" or "SOURCE" directories
   in common repositories
http://download.fedora.redhat.com/pub/fedora/linux/core/5/source/SRPMS/

2. As root, unpack the source RPMs
   rpm -ivh example.src.rpm

.... this will put the patches in /usr/src/redhat/SOURCES. But these
are not the real working file.
.... there is also a build *.spec file put in /usr/src/redhat/SPECS
 
3. Build the source code of the packages using:
 command: rpmbuild -bp --target=<arch> kernel.spec
      # cd /usr/src/redhat/SPECS/
      # rpmbuild -bp --target=i686 kernel-2.6.spec
.... the source may appear in /usr/src/redhat/BUILD

4. Modify and recompile source
5. Package into rpm using:

Linux Commands
===============

XConfuration:
/usr/sbin/mouseconfig  ------- mouse
chsh ------------------------- to change default shell
grep chee /etc/passwd -------- tells you what shell you are from

Xconfigurator ---------------- to use XFree86
/etc/X11/XF86Config ---------- configuration file
ls /usr/lib/X11/rgb.txt ------ to see a list of X11 defined color names


/usr/sbin/timeconfig --------- change time

/usr/sbin/ntsysv ------------- choose program to
/sbin/chkconfig -------------- boot during startup

/usr/doc/HOWTO --------------- HowTo help files


Commands:
useradd **** ----------------- add user with name ****
passwd ***** ----------------- make password for user
pwd -------------------------- print working directory
su **** ---------------------- to change to another user (from superuser)
su --------------------------- superuser
/sbin/shutdown -r now -------- shutdown and reboot now
startx ----------------------- to start X windows
ps -e/f/a/l/u ---------------- process
mount /dev/fd0 /tmp ---------- mount 1st floppy drive to /tmp
umount /tmp ------------------ umount any device from /tmp
cat notes >> notes2 ---------- prints the contents of a file to standard output
      and then redirects and APPEND to file "notes2"
head [-count] *****       list first few lines of file
tail [-/+ [startline] [bcl]] ***  list the last few lines
wc **** ---------------------- count the number of words, lines etc in a file
alias l='ls -al' ------------- make an alias, eg. to list files
ln **file1** **file2** ------- link file2 to original file 1 (hard link)
ln -s **file1** **file2**----- do symbolic link
ls -i ------------------------ list file and look and their i-node to see linked files
rsync -auv src dest ---------- copy / update between local/remote src/dest
size ------------------------- to calculate the size of the program
wc -l/c/w **file** ----------- Word Count lines, characters, words in a file
tr --------------------------- Translate, delete or replace text in a file.
tar cf - . | zip name -  ***** Tar and then zip
unzip -p backup | tar xf - *** reverse of above - ie. unzip and untar
script <filename> ------------ starts a shell - records the whole sesion - exit the shell to stop script


Help Commands:
man -k **** ------------------ to search for keywords in manual entries
man # intro ------------------ #={1..8}, 8 intro topics of Unix manual.
man -M /opt/lam/man mpirun --- manual for "mpirun" program installed in /opt/lam
rpm -qdf /usr/bin/rtin ------- query
zless /usr/doc/HOWTO/...gz --- read compressed howto files without uncompressing
locate -d <dbfile> <filename>- find a command or file called filename
updatedb --localpath=<dirPath> --output=<dbfile>
                               to update data file before using locate
                               dirPath=path of file system to put in db
                               dbfile=database file to log info to.
slocate ......                 replaces updatedb and locate commands
/etc/cron.daily/update ------- create database before using locate
find / -name .... ------------ find the file named ....
find ~ -size 0 -print -------- find file with zero size
crontab -e ------------------- to edit the crontab for current user

Build / Compile Programs
./configure --prefix=/home/che sets the installation to your local dir so no need root permissions

Environment:
set -------------------------- to list enviroment variables
xport AUTO=VW ---------------- set the variable auto=vw
alias ls='ls -color=auto' ---- colorize ls .. put this in .profile or .bashrc

./a.out ---------------------- to execute the file "a.out"
. <command/script> ----------- execute command or script in CURRENT environment

Process Control:
kill -9 PID ----------------- sure kill the process
nohup **** ------------------ No hang-up for process ****
nice -# **** ---------------- reduce priority of command
nice --# **** --------------- increase priority of command
sleep **** ------------------ timer - stalls for certain time
at -l/r/m time date **** ---- timer - run command AT specified TIME

Compilers
cc -------------------------- SGI MIPS c compiler
CC -------------------------- SGI MIPS c++ compiler
gcc ------------------------- gnu c compiler
g++ ------------------------- gnu c++ compiler
insight --------------------- online help manual on SGI MIPS C/C++ compilers

Gnuplot
load "plotfile" ------------- runs the script file "plotfile"
set term postscript color
set output "file.ps" -------

Emacs
<shift> <left-mouse> -------- change font size
.Xdefaults ------------------ change emacs properties
M-x replace-strings --------- to replace all occurances of strings
C-q C-m --------------------- produces the letters "^M"
M-x replace-strings, C-q C-m  dos2unix file conversion within emacs
C-c C-w --------------------- Fortran End of Column marker
C-c C-r --------------------- Fortran ruler
M-C-j ----------------------- Fortran continuation line


Browsers:
netscape -install ----------- make colour look better

Graphics
gif2ppm --------------------- convert gif to portable pixmap format
pnmtogif -------------------- convert any pixmap to gif
pstoimg -transparent -------- make background transparent

Secure Operations
scp newfile chee@parallel.hpc.unsw.edu.au:~/programs

Networking
nslookup www.google.com ----- returns the IP address of the URL
ping ------------------------
ifconfig <ethernet card> ---- configuration of adapter
netstat -in -----------------
echo 1 >/proc/sys/net/ipv4/ip_forward -- allow routing
echo 1 >/proc/sys/net/ipv4/conf/all/rp_filter -- prevent IP spoofing

System Admin / Setup
/etc/passwd ----------------- user details
/etc/sys_id ----------------- name of the system
/etc/hosts ------------------ list of IP address and Names
~/.rhosts ------------------- list of computers and usernames for rsh without password
background ------------------ customizing background of screen
ls -l /lib/libc-*  ---------- check the glibc version
uname -r   ------------------ check the kernel version
/etc/rc.d/rc.local ---------- startup programs are included here

System /Hardware Information
lsmod ----------------------- list loaded modules
lsdev ----------------------- display info about installed hardware
modinfo --------------------- information about installed modules
free ------------------------ display free/used memory on the system
more /proc/interrupts ------- view IRQ channels
more /proc/dma -------------- view dma channels
more /proc/ioports ---------- view I/O memory addresses

Change Resources
ulimit -a ------------------- check user limit
ulimit -aH ------------------ check user HARD-limit
ulimit -s 32678 ------------- set SOFT & HARD stack limit
ulimit -sS 32678 ------------ set SOFT stack limit
echo 8192 >/proc/sys/fs/file-max   --- increase number of files openable
echo 16384 >/proc/sys/fs/inode-max --- increase number of files openable

File Commands
file ***** ------------------- prints the type of the file ****
find -c ---------------------- find file which has changed
stat -t ---------------------- provides info about file
diff file1 file2 ------------- output differences between files

Remote X Gui Interface
startX ----------------------- starts the X server
startkde --------------------- starts the KDE desktop

Modules
insmod ----------------------- insert module quick UNSAFE way
modprobe --------------------- insert module SAFE way