Sunday, December 06, 2009

Using SQL Connection Dialog

This article shows how to include the SQL Connection Dialog into your code in C#. There is quite a good article with many google hits on this subject at:
http://www.codeproject.com/KB/vb/SQL_Connection_Dialog.aspx?msg=2320576
However, this is written in VB.

There is also a less well known article on this subject written in C#.
http://www.gotdotnet.ru/forums/1/94733/451052/#post451052 (in C#)
However, this lacks the Test Connection button and the overall structure was not very clear.


Yet having the two article above and numerouse searches in between, it still took me a few hours to get the SQL Connection Dialog working properly. So I thought I'll publish my own C# code here. Although similar to the two references, I were a few key points which I need to change to make it work. Also I hope to show that my code structure is clearer in the relationship between the form and the SQL Connection dialog.



There are 3 separate code files below and I've developed in Visual Studio 2005. First Create a C# Windows forms project.
1. Program.cs - this is the executable that calls on Form1 class. This is automatically generated.

2. Form1.cs - this is also autogenerated. But I added my own button called btnSQLconnect (using drag and drop to drag the Button from the Toolbox).
In Form1.cs, I also added the method btnSQLconnect_Click_1 that will be called when the button is pressed.
Basically the idea is you have your own form, and you have a button on that form to allow the user to click and that will open up the SQL Connection dialog.

3. SQLconnectForm.cs - this is the main thing to add to make use of the SQL Connection Dialog. The rest of this article is devoted to explaining this in greater detail.

Firstly, note the SQL Connection Dialog, need to be a Windows Form by itself, hence the creation of the SQLconnectForm.cs class (which inherits from Form) to hold it.

Secondly, we need to crucial components to make the SQLConnectionForm to become the actual SQL Connection Dialog:
SqlFileConnectionProperties cp;
SqlConnectionUIControl uic;

The cp contain the connection properties to connect to the DB. The uic is the .Net built-in SQL Connection Dialog.

A class member called sConnectionString is defined as the get / set method for the actual connection string stored within cp.

The initialization of the SQL Connection Dialog is done in the constructor of the SQLconnectionForm class. Initialization includes:
i) Setting values for properties of cp and uic.
ii) Creation of 4 buttons: OK, Cancel, Test Connection, Advanced.
iii) Setting properties (including position) of the 4 buttons.

The Advanced and Test Connection button cannot be seen on the visual designer of SQLConnectionForm, but they are definitely there when the program is run.
The OK and Cancel buttons are added in the usual way by dragging them from the Toolbox onto the Form in the Visual Designer.

Each of the 4 buttons also have four methods to handle the event when the buttons are clicked.

The Advanced button clicked event will call another form which has been predefined in the .Net. Hence there is only a few lines of code in the Avanced's click method.

The Test Connection event method is where an attempt is made to open and close the DB to check that the connection is OK. Initially, there was an error like "unable to login user" as it tries to connect to the DB. The solution that worked is to modify the User Instance in the connection string as:
cp.ConnectionStringBuilder["User Instance"] = false;

If you have similar problems, consider commenting out the line above.



Perhaps the most challenging concept the SQL Connection Dialog (at least for me) is that we need to create a blank Form (SQLconnectionForm.cs), then by including the components uic and cp, the form magically becomes the standard Microsoft .Net SQL Connection Dialog.


*********** Program.cs *****************************
using System;
using System.Collections.Generic;
using System.Windows.Forms;

namespace MapDB2Class_GUI
{
static class Program
{
///
/// The main entry point for the application.
///

[STAThread]
static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new Form1());
}
}
}

*********** Form.cs **********************
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using Microsoft.Data.ConnectionUI;


namespace MapDB2Class_GUI
{
public partial class Form1 : Form
{

public Form1()
{
InitializeComponent();
}

private void btnSQLconnect_Click_1(object sender, EventArgs e)
{
SQLconnectForm sdlg = new SQLconnectForm("Selecting DB");
sdlg.ShowDialog();
}


}
}

*************** SQLconnection.cs *************************
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using Microsoft.Data.ConnectionUI;
using System.Data.SqlTypes;
using System.Data.SqlClient;

namespace MapDB2Class_GUI
{
public partial class SQLconnectForm : Form
{

SqlFileConnectionProperties cp;
SqlConnectionUIControl uic;

public string sConnectionString
{
get { return cp.ConnectionStringBuilder.ConnectionString; }
set { cp.ConnectionStringBuilder.ConnectionString = value; }
}


public SQLconnectForm(string sTitle)
{
// This call is a default for Windows Form
InitializeComponent();

this.Text = sTitle; // title of this form
this.AcceptButton = btnOK;
this.CancelButton = btnCancel;


this.Padding = new Padding(5);
uic = new SqlConnectionUIControl();
Button adv = new Button();
Button tst = new Button();

cp = new SqlFileConnectionProperties();
uic.Initialize(cp);
uic.LoadProperties();
uic.Dock = DockStyle.Top;
uic.Parent = this;


this.ClientSize = Size.Add(uic.MinimumSize, new Size(10, adv.Height + 25));
this.MinimumSize = this.Size;

adv.Text = "Advanced";
adv.Dock = DockStyle.None;
adv.Location = new Point(uic.Width - adv.Width, uic.Bottom + 10);
adv.Anchor = AnchorStyles.Right | AnchorStyles.Top;
adv.Click += new EventHandler(Advanced_Click);
adv.Parent = this;

tst.Text = "Test Connection";
tst.Width = 100;
tst.Dock = DockStyle.None;
tst.Location = new Point((uic.Width - tst.Width) - adv.Width - 10, uic.Bottom + 10);
tst.Anchor = AnchorStyles.Right | AnchorStyles.Top;
tst.Click += new EventHandler(Test_Clicked);
tst.Parent = this;



}

void Test_Clicked(object sender, EventArgs e)
{
cp.ConnectionStringBuilder["User Instance"] = false; // Need to change this to false, otherwise cannot login user.
SqlConnection conn = new SqlConnection(cp.ConnectionStringBuilder.ConnectionString);
try
{
conn.Open();
MessageBox.Show("Test Connection Succeeded.", "Connected", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
}
catch (Exception exc)
{
MessageBox.Show("Test Connection Failed.", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
finally {
try { conn.Close(); }
catch (Exception exc) { }
}
}

void Advanced_Click(object sender, EventArgs e)
{
Form frm = new Form();

PropertyGrid pg = new PropertyGrid();
pg.SelectedObject = cp;
pg.Dock = DockStyle.Fill;
pg.Parent = frm;

frm.ShowDialog();
}

private void btnOK_Click(object sender, EventArgs e)
{
this.DialogResult = DialogResult.OK;
this.Close();
}

private void btnCancel_Click(object sender, EventArgs e)
{
this.DialogResult = DialogResult.Cancel;
this.Close();
}

}
}

Saturday, November 14, 2009

Notes Intel Fortran


NotesIntelFortran
=====================

Contents
=========
msvcrtd.dll issue
dumpbin, editbin, stack
Fortran routines for DLL
Example Fortran routines for DLL called by C#
Fortran C# wrappers and data compatibility
Setup IMSL
Setup MKL
Setup - Intel Fortran / IMSL Environment Variables
Intel Fortran 10.1 and VS. Net 2005
Intel Fortran 11.0
Managed Code
BLAS, IMSL and MKL
Building DLLs (Fortran DLLs used in Fortran apps)
!DEC$ ATTRIBUTES directives
Passing module variables and functions in DLL
Best Practice
Errors - Debugging
How to Add Version and other Metadata to DLL or EXE
Using VTune
Compiler Options
Build Macros (eg $(OUTDIR))
Using MKL
Using LAPACK95 & General Comment on DLLs, LIBs, Mod files
Mixed language programming
Stack Checking
Enable Vectorization and Report
Enable OpenMP
Using Thread Profiler
Using Thread Checker
Profile Guided Optimization
Using code coverage


msvcrtd.dll issue
====================

***************
Performance Tools for Software Developers
libmmdd.dll is dependent on msvcrtd.dll which is no longer distributed.

Symptom(s):
Note: This only applies to the compilers for Intel® Extended Memory 64 Technology and for the Itanium® Architecture.
Applications or DLL's that are built with /MDd or directly link against Intel's libmmdd.dll may emit the runtime error.
This application has failed to start because msvcrtd.dll was not found. Re-installing the application may fix this problem.

Cause:
The Platform SDK distributed with Microsoft* Visual Studio* 2005 does not contain msvcrtd.dll. Using /MDd links against the Intel math library libmmdd.dll which has a dependency on msvcrtd.dll.

Solution:
This is a known issue that may be resolved in a future product release. As a work-around, use the msvcrtd.dll distributed with the Microsoft* Platform SDK available at http://www.microsoft.com/downloads/details.aspx?FamilyId=0BAF2B35-C656-4969-ACE8-E4C0C0716ADB&displaylang=en  † .
***************

May need to get msvcrtd.dll from somewhere to be put into c"\windows\system32"




dumpbin, editbin, stack
========================
To run these command line tools,
- go to "Start" button -> "All Programs"
  -> "Intel Software Development Tools"
  -> "Intel Compiler 8.0"
  -> "Build Environment for Fortran IA-32 Applications"

To check the stack size of a program.
Run "dumpbin /headers executable_file", and you can see the "size of stack reserve" information in "optional header values".

To enlarge the stack of a program:
Run "editbin /STACK: program.exe"



Alternatively
http://www.atalasoft.com/cs/blogs/rickm/archive/2008/04/22/increasing-the-size-of-your-stack-net-memory-management-part-3.aspx
The Easiest Way ( .NET 2.0 )
In .NET 2.0 and newer you can simply specify thread size in a thread’s constructor. Unfortunately, this method is only compatible only with Windows XP and newer operating systems. You can specify this parameter on those platforms but it will have no effect; the stack size in the binary header will be used.
using System.Threading;

Thread T = new Thread(threadDelegate, stackSizeInBytes);
T.Start();


Fortran routines for DLL
=========================

  Interface
    subroutine my_sub(I)
        !DEC$ ATTRIBUTES C, ALIAS:"_My_Sub" :: my_sub
        integer i
    end subroutine
  end interface


- Case Sensitive: Fortran is not, C/C++ is.
- Arrays are always passed by reference
- ATTRIBUTES for a routine may be: C, STDCALL, REFERENCE, VARYING
- ATTRIBUTES for an argument may be: VALUE, REFERENCE
- C or STDCALL makes passing all arguments by value, except arrays.
- the VALUE or REFERENCE argument options, overide the routine option
  of C or STDCALL.
- for IA-32 system, need to put underscore for routine to be called by C.
- cannot call internal procedures from outside the program unit that contains them.
- To pass Variable number of arguments, need C and VARYING, not STDCALL


Example Fortran routines for DLL called by C#
================================================
! Public wrapper for status_msg_get_code
integer*4 pure function StatusMsgGetCode(msg)
!DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_StatusMsgGetCode' :: StatusMsgGetCode
!DEC$ ATTRIBUTES REFERENCE :: msg
    StatusMsgGetCode = status_msg_get_code(msg)


- uses STDCALL (can't handle optional argument (see Intel F User Guide)
- uses alias with leading underscore
- wrap and rename code to get rid of underscore in function name, eg.
    status_msg_get_code  --> StatusMsgGetCode
- uses REFERENCE to pass arguments

Fortran C# wrappers and data compatibility
=============================================
This is best illustrated by example:

Fortran function in Fort.dll:
    subroutine Foo_dll()
    !DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_Foo_dll'  :: Foo_dll
    !DEC$ ATTRIBUTES REFERENCE ::
.........
    end subroutine

C# declaration
#if x64
        [DllImport("Fort.dll", EntryPoint = "_Foo_dll")]
#else
        [DllImport("Fort.dll", EntryPoint = "Foo_dll")]
#endif
        private static extern void Foo_dll();

public static void CallFoo_dll()
        {
........
Foo_dll();
        }

Below lists the Fortran to C# data type declarations, with X as the variable name:
Fortran C# C# C#
integer(4) X [In, Out] ref int    X ref int      X ref X
real(8)    X [In, Out] ref double X ref double X ref X
real(8)    X(N)     [In, Out] double[] X ref double[] X              X



Setup IMSL
===========
Documentation -
1. Start -> Programs -> IMSL Fortran Library 5.0. This contains:
QuickStart, Readme, User's Guide
2. PDF docs contains
Math Library V1, V2, Statistical Libraries, Special Functions

IMSL is not Thread safe. It is still safe to use, provided that calls to the
IMSL routines are made from a single thread.

VS.Net integration
1. In VS.Net, goto Tools -> Options -> Intel(R) Fortran -> Project Directories ->
type in the Include and Libraries directory path.
2. Specify the following include statements;
   include 'link_f90_static.h'
   include 'link_f90_dll.h'
   include 'link_f90_static_smp.h'
   include 'link_f90_dll_smp.h'
or go to Projects -> Add Existing Item ... browse to add the library.

The link*.h files contain directives to point to certain *.dll files. For example,
link_f90_dll.h contents are:
!dec$objcomment lib:'imsl_dll.lib'
!dec$objcomment lib:'imslscalar_dll.lib'
!dec$objcomment lib:'imslblas_dll.lib'

3. Inside the code, in addition to the include directives in step 2, need to include
some USE statements. For example, to use the random number generator rnun, we need:
i) use rnun_int; or
ii) use imsl_libraries; or
iii) use numerical_libraries

iii - is used to provide backward compatibility with previous IMSL libraries and Fortran77
version of the library. It may not be necessary to use iii and calling the functions as before
will continue to work.

Using ii provides access to all the IMSL functions, so individual use statements are not needed.
However, some may choose to use i because it shows explicitly which functions are called.

Using BLAS
1. Intel MKL Blas library used automatically when IMSL is linked with the
SMP (ie. multiprocessing) option.
2. See ia32 or ia64 Readme to link 3rd party blas with IMSL.

IMSL version 6.0
- IMSL is now THREAD SAFE
- Env Var - run ia32\bin\fnlsetup.bat .
- MUST remove old references, eg. include 'link_f90_dll.h'   (because new headers have diff name)
- MUST rename directory of older installations of IMSL, so that any old env vars cannot
  accidentally point to it.
- Add include statement in the relevant source files:
     include 'link_fnl_shared.h'       ! for dynamic dlls
include 'link_fnl_shared_hpc.h'   ! for dynamic dlls and SMP (OpenMP)
- Add include directory in VS.Net
    Project - Properties - Fortran - Include Directories: $(FNL_DIR)\ia32\include\dll
- Add library directory in VS.Net
    Project - Properties - Fortran - Library Directories: $(FNL_DIR)\ia32\lib
- Run the ASSURANCE tests provided by IMSL in ...\examples\eiat. Note that
  in run_test.bat, need to use %LINK_FNL_STATIC%


Setup MKL
==========
Linking to MKL can be done either statically *.lib or dynamically *.dll

For ia32 apps, when linking statically, link to mkl_c.lib or mkl_s.lib
For ia32 apps, when linking dynamically, link to these STATIC libs:
   mkl_c_dll.lib or mkl_s_dll.lib
that will provide interfaces to the correct DLLs.

For MKL v 10.0
- Major changes, MKL divided into layers: Interface, Threading, Computation, RTL.
- Support for 64bit via ILP64/LP64.
- Use of OpenMP for threading and MPI and Scalapack for distriubuted computing.
- Env Vars: The following variables would have been set by running tool/environment/mklvars32.bat
  $(MKLPATH) = root location of MKL directory - eg D:\programs\Intel\MKL\10.0...
  lib=$(MKLPATH)\ia32\lib
  include=$(MKLPATH)\include
  bin=$(MKLPATH)\ia32\bin
  LIBRARY_PATH=$(MKLPATH)\ia32\lib
  CPATH=$(MKLPATH)\include
  FPATH=$(MKLPATH)\include
- Visual Studio config
  Project -> Properties -> Linker -> General -> Add additional Library Directories
       $(MKLPATH10)\ia32\lib
  Project -> Properties -> Linker -> General -> Add additional Include Directories
$(MKLPATH10)\include
$(MKLPATH10)\interfaces\lapack95

- Linking
  Intel advises to link libguide and libiomp dynamically even if others are linked statically.
  Link items to consider:
      Interface:            Threading:              Computation:         RTL:         Description:
 mkl_intel_c_dll.lib   mkl_sequential_dll.lib      mkl_core_dll.lib           Dynamic, non-parallel, 32bit

  Actual linking done in code by using the !$dec attributes such as:
        !dec$objcomment lib:'mkl_intel_c_dll.lib'
   !dec$objcomment lib:'mkl_sequential_dll.lib'  
!dec$objcomment lib:'mkl_core_dll.lib'
   !dec$objcomment lib:'mkl_lapack95.lib'
- You are advised to link with libguide and libiomp dynamically even if other libraries are
linked statically. (MKL user guide, Chap 5)

- To use THREADED / PARALLEL / OPENMP Intel MKL, it is highly recommended to compile your code with the /MT
option. The compiler driver will pass the option to the linker and the latter will load
multi-thread (MT) run-time libraries.
- For multi-threading based on Intel OpenMP
      Interface:          
lib\mkl_intel_c_dll.lib
 Threading:        
lib\mkl_intel_thread_dll.lib,
bin\mkl_intel_thread.dll,  
 Computation:        
lib\mkl_core_dll.lib,
(many many bins....)
 RTL:        
lib\libguide40.lib, OR lib\libiomp5md.lib,
bin\libguide40.dll, OR bin\libiomp5md.dll









Setup - Intel Fortran / IMSL Environment Variables
=====================================================
user defined:
INCLUDE
C:\Program Files\VNI\CTT5.0\CTT5.0\INCLUDE\IA32;C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\include\
LIB
C:\Program Files\VNI\CTT5.0\CTT5.0\LIB\IA32;C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Lib\
PATH
C:\Program Files\VNI\CTT5.0\CTT5.0\LIB\IA32;%PATH%;d:\DATA\UsercheeOnD\tools\NixTools\bin;%MSNET_C%\bin;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin;%PStill%;D:\Program\UnderstandF90\bin\pc-win95

system variables:
CTT_DIR
C:\Program Files\VNI\CTT5.0\CTT5.0\LIB\IA32;%PATH%;d:\DATA\UsercheeOnD\tools\NixTools\bin;%MSNET_C%\bin;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin;%PStill%;D:\Program\UnderstandF90\bin\pc-win95
CTT_EXAMPLES
"C:\Program Files\VNI\CTT5.0\CTT5.0\examples\IA32"
CTT_FORTRAN_COMPILER
Intel(R) Fortran Compiler for 32-bit applications, Version 8.1
CTT_OS_VERSION
Microsoft Windows XP/2000/2003
F90
ifort
F90FLAGS
/w /I:"C:\Program Files\VNI\CTT5.0\CTT5.0\include\IA32" /fpe:3 /nologo
FC
ifort
FFLAGS
/w /I:"C:\Program Files\VNI\CTT5.0\CTT5.0\include\IA32" /fpe:3 /nologo
FP_NO_HOST_CHECK
NO
INCLUDE
C:\Program Files\VNI\CTT5.0\CTT5.0\INCLUDE\IA32;%INTEL_FORTRAN80%\ia32\include;C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\include\
INCLUDE_DIR
"C:\Program Files\VNI\CTT5.0\CTT5.0\include\IA32"
INTEL_FORTRAN80
C:\Program Files\Intel\Fortran\Compiler80
INTEL_LICENSE_FILE
C:\Program Files\Common Files\Intel\Licenses
KMP_DUPLICATE_LIB_OK
TRUE
LIB
C:\Program Files\VNI\CTT5.0\CTT5.0\LIB\IA32;%INTEL_FORTRAN80%\ia32\lib;C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Lib\
LIB_ARCH
IA32
LINK_F90
imsl_dll.lib imslscalar_dll.lib imslblas_dll.lib
LINK_F90_DLL
imsl_dll.lib imslscalar_dll.lib imslblas_dll.lib
LINK_F90_DLL_SMP
/Qopenmp /F6000000 /fpp imsl_dll.lib imslsmp_dll.lib mkl_c_dll.lib /link /nodefaultlib:libc.lib
LINK_F90_SMP
/Qopenmp /F6000000 /fpp imsl_dll.lib imslsmp_dll.lib mkl_c_dll.lib /link /nodefaultlib:libc.lib
LINK_F90_STATIC
imsl.lib imslscalar.lib imslblas.lib imsls_err.lib
LINK_F90_STATIC_SMP
/Qopenmp /F6000000 /fpp imsl.lib imslsmp.lib mkl_c_dll.lib imsls_err.lib /link /nodefaultlib:libc.lib
Path
d:\Program\Intel\VTune\CGGlbCache;d:\Program\Intel\VTune\Analyzer\Bin;d:\Program\Intel\VTune\Shared\Bin;C:\Program Files\PC Connectivity Solution\;c:\program files\vni\ctt5.0\ctt5.0\lib\ia32;%systemroot%\system32;%systemroot%;%systemroot%\system32\wbem;c:\program files\ibm\trace facility;c:\program files\personal communications;c:\program files\ati technologies\ati control panel;c:\program files\common files\adaptec shared\system;c:\program files\ibm\trace facility\;c:\program files\intel\fortran\idb80\bin;%intel_fortran80%\ia32\bin;c:\program files\host integration server\system;c:\program files\ibm\personal communications\;c:\progra~1\ca\shared~1\scanen~1;c:\program files\ca\sharedcomponents\scanengine;c:\program files\ca\sharedcomponents\caupdate\;c:\program files\ca\sharedcomponents\thirdparty\;c:\program files\ca\sharedcomponents\subscriptionlicense\;c:\progra~1\ca\etrust~1;C:\Program Files\MATLAB\R2007a\bin;C:\Program Files\MATLAB\R2007a\bin\win32;C:\Program Files\Common Files\Roxio Shared\DLLShared\;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE

VNI_DIR
C:\Program Files\VNI\CTT5.0\CTT5.0\..
VNI_F90_MSG
C:\Program Files\VNI\CTT5.0\CTT5.0\BIN\IA32


Intel Fortran 10.1 and VS. Net 2005
=====================================
Manually add this to SYSTEM VARIABLE -> Path from Control Panel

D:\Program\VNI\imsl\fnl600\IA32\LIB;
C:\Program files\MPICH2\bin;
D:\Program\Intel\Compiler\Fortran\10.1.013\Ia32\Bin;
C:\Program Files\Common Files\Intel\Shared Files\Ia32\Bin;
D:\Program Files\Microsoft Visual Studio 8\Common7\IDE;
D:\Program Files\Microsoft Visual Studio 8\VC\BIN;
D:\Program Files\Microsoft Visual Studio 8\Common7\Tools;
D:\Program Files\Microsoft Visual Studio 8\Common7\Tools\bin;
D:\Program Files\Microsoft Visual Studio 8\VC\PlatformSDK\bin;


Manually add this to SYSTEM VARIABLE -> Lib from Control Panel
C:\Program files\MPICH2\LIB;%IFORT_COMPILER10%Ia32\Lib;%MSVS8%\VC\atlmfc\lib;%MSVS8%\VC\lib;%MSVS8%\VC\PlatformSDK\lib;%FNL_DIR%\IA32\lib;



Intel Fortran 11.0
===================
1. New: Floating Point Model, some are not compatible with Floating Point Speculation
2. New: OpenMP 3.0 standard included
3. New: Fortran 2003 features included
4. Some functions may fail -> use macro like CBAEXPMODTEST=1 to mark out certain things.
5. See Fortran User / Ref Guide -> Building Apps -> Using Libraries -> Using IMSL
6. IMSL Readme.txt -> KAPMR does not behave in thread safe manner.
        Use OpenMP critical region around KAPMR to be safe.









Managed Code
=============
Mixed-Language Programming and Intel Visual Fortran Project Types
This version of Intel Visual Fortran produces only unmanaged code, which is architecture-specific
code. You cannot create an Intel Visual Fortran main program that directly calls a
subprogram implementing managed code. To call managed code, you can call an unmanaged
code subprogram in a different language that does support calling managed code.


BLAS, IMSL and MKL
===================
Blas is implemented by IMSL - details are found in Chapter 9: Basic Matrix/Vector Operations.
Blas is also implemented by the hardware vendor - in this case Intel - in Intel's MKL library,
which may be written in machine code.

The BLAS API, i.e. the calling convention of the routines, are the same whether they are
implemented by MKL or IMSL. For example, SDOT is the routine that finds the dot product of two
vectors.

To use different implementation, the program has to link with different libraries.
For IMSL: imslblas_dll.dll
For MKL: mkl_p4.dll

By default, when using link_f90_dll.h, it include's IMSL's BLAS (see section "Setup IMSL")
By default, when using link_f90_dll_smp.h, it include's MKL's BLAS (see section "Setup IMSL")

If we want to use MKL without the SMP (parallel processing) feature, then instead of using
link_f90_dll.h, we have to manually add the directives and point to the correct BLAS, eg:

!dec$objcomment lib:'imsl_dll.lib'
!dec$objcomment lib:'imslscalar_dll.lib'
!dec$objcomment lib:'mkl_ia32.lib'

The DLL (*.dll) can be placed anywhere the system knows of, eg:
c:\windows\system32\ mkl_def.dll, mkl_p3.dll, mkl_p4.dll
(IMSL provides these 3 dlls from the MKL package)

The mkl_ia32.lib contain STATIC INTERFACES to dlls including BLAS, cblas, FFTs, VML.
However, there is no corresponding single mkl_ia32.dll. Instead it is spread over a few DLLs,
such as mkl_def.dll, mkl_vml_def.dll, mkl_lapack32.dll, etc.

If a function (eg vsinv from VML package of MKL) is included in the library mkl_ia32.lib,
but the dll does not exist, then the code WILL COMPILE. But during runtime, a fatal error
would occur because it cannot find and use the dll.

NOTE: the IMSL dlls and libs are installed in
C:\Program Files\VNI\CTT5.0\CTT5.0\lib\IA32



Building DLLs (Fortran DLLs used in Fortran apps)
=================================================

Note:
When a DLL is built the output are two files:
1) *.dll - has the library's executable code
2) *.lib - the import library providing interface between the program and the dll.

The notes here presents two cases:
Case A: DLL to be created in its own separate VS solution, called solnA, in project projA.
        The two generated output will be projA.dll and projA.lib
Case B: DLL to be created in a project (projB) in the same solution (solnB) , as the
        application project (projC).
(The application project contains the code that uses the DLL.)
The two generated output will be projB.dll and projB.lib


1. Build DLL project in its own solution
- Say we call this Solution solnFoo, and Project projFoo

Case A:
- From VS.Net - in new solution, create a new DLL project by:
  File -> New -> Project -> Intel Fortran projects -> Dynamic link library

Case B:
- From VS.Net - in existing solution, create a new DLL project by:
  File -> New -> Project -> Intel Fortran projects -> Dynamic link library


2. Write a subroutine and expose it, eg:
subroutine hello()
  !DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_hello' :: hello
    (do blah blah)
end subroutine hello

- put this subroutine by itself into a file (eg. hello.f90) or into
a module (eg hello_mod.f90)

- DLLEXPORT needed to expose the name of the routine
- alias is needed for compatibility with Intel Fortran and VS.NET environment


3. Build the DLL in VS.NET by:
- Build (menu) -> Build or Build Solution
- Copy the *.lib and *.dll files and put them into same directory as the
executable code for the application; i.e. same directory as projC.exe

4. Link the DLL via the lib file by:
- Go to the application project "program" file or "module" file and put this near the
start of the file:
CASE A:       !dec$objcomment lib:'projA.lib'
CASE B:       !dec$objcomment lib:'projB.lib'

CASE B only:
- Ensure that the dependencies eg projB is UNchecked in the Project Dependency dialog box of ProjC

- in the solution explorer in VS.NET, click on the application's
project name, eg projC.
- From the Project menu or right clicking on the project, go to "Add existing item ..."
- Browse and choose "projB.lib" to add. The lib file should appear under solution explorer.
- From the Project menu or right clicking on the project, go to "Project Dependencies..."
- Alternative to the "Add Existing item..." way is to specify through the linker by:
  with the project name highlighted, go to Project menu -> Properties -> Linker
  -> "Additional Library Directories" -> type in dir path where *.lib is located.

5. Add interface to DLL routine in the application.
- goto into the subroutine of the application and add the following:

module projC_app
    contains
    subroutine app()
interface
            subroutine hello()
            !DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_hello' :: hello
            end subroutine hello
        end interface
     end subroutine
end module

- DO NOT ADD the interface on the top level, eg DO NOT add in the starting part of a module. Instead
add the interface inside the module's subroutine that makes the call to the DLL routine.

- compile and run. Ensure that building mode is RELEASE, not DEBUG.


!DEC$ ATTRIBUTES directives
============================
1. C vs STDCALL - for controlling the stack of passed variables.
- both of these will try to make variables pass by value, rather than the Fortran default of
passing by reference.
- arrays are always passed by reference
- C -> the calling routine controls the stack. larger code.
- C -> possible to call variable number of arguments, MUST use "C, VARYING" to let
  Fortran know that multiple arguments are called.
- C -> is default in C/C++ code. to use with fortran code, either
  i) change the c code to STDCALL; or
     extern void __stdcall foo_f90(int n);
  ii) change the f90 code to use C convention
     !DEC$ ATTRIBUTES C :: foo_f90
- STDCALL -> the called routine controls the stack.

2. VALUE vs REFERENCE
- for fortran, C or STDCALL will change default to passing by value, except arrays which will
  be passed by reference
- But, each argument of the subroutine can be declared with VALUE or REFERENCE to override the
  default mode, eg:
     subroutine foo(a, b)
     !DEC$ ATTRIBUTES VALUE :: a
     !DEC$ ATTRIBUTES REFERENCE :: b


Passing module variables and functions in DLL
==============================================
Consider passing the variable 'foo' and calling method fooA() defined in a module 'mod_foo'

1. Expose the variable foo and fooA()
!DEC$ ATTRIBUTES DLLEXPORT :: foo
!DEC$ ATTRIBUTES DLLEXPORT :: fooA

Do not use ALIAS.

2. Build and Copy the following files from the DLL build directory to the application directory.
mod_foo.dll, mod_foo.lib, mod_foo.mod

3. In the application that uses 'foo', add the statement:
use mod_foo

This technique is only useful when both application and DLL are written in Fortran. The variable
names will have leading underscore "_". This is transparen to the user who uses "use mod_foo".
Such DLL are not convenient for DLLs that are to be used with other languages because of the leading
underscore on variable names.


Best Practice
==============
1. For optimized code:
- use /fast
- use "Configuration Properties -> Fortran -> Optimization -> Require Intel Processor Extension

2. To check for stack overflow
- /Qfpstkchk
- /Ge, /Gsn, /Fn

3. Fortran DLL structure
- Put constant data into a module, say mod_consts, and expose to data as:
!DEC$ ATTRIBUTES DLLEXPORT :: eps4
  Note: do not ALIAS
- Put subroutines into another module, say mod_funcs and expose data:
use mod_consts
subroutine blah()
!DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_testInt4'  :: testInt4
  Note: use alias so it is accessible outside
- Construct interface modules for application:
        module interface_mod
   use mod_consts
   interface
subroutine blah()
!DEC$ ATTRIBUTES DLLEXPORT, STDCALL, ALIAS:'_testInt4'  :: testInt4
      ............
- Include interface in the application
         use interface_mod

This technique allows other Fortran projects to make use of both data and functions in DLLs.
However, other languages will not be able to make use of the data directly (may need to have underscore
for variable names in the other languages calling this Fortran DLL).


Errors - Debugging
===================

General Sources:
"List of Run-Time Error Messages", Intel Visual Fortran compiler doc
- from Building Applications -> Error Handling -> Handling Run Time Errors ->

Cryptic LNK errors
1. When using a function from another place, eg DLL, etc; ensure that an "interface" block is written
for at the code which calls the function.
2. Ensure the library path is defined. Eg. In VS.Net -> right click project -> Properties -> Linker
-> General ->  "Additional Library Directories"

Access Violation
1. Passing integer*4 into a subroutine with parameter declared as integer*8
2. Subroutine A in a module is DLL exported. Another subroutine within the same project uses subroutine A from another module WILL cause a CONFLICT. Since it is being used within the same project, subroutine A need a wrapper which is NOT DLL exported. This wrapper can be called by other module subroutines within the same project.
3. When an ARRAY of derived type contains components which are also derived types, then it must be declared with fixed size (i.e. hardcode dimension) or the variable must be a dynamic array (i.e. declared ALLOCATABLE). It cannot be declared with size specified by a parameter.
eg.
function foo(a, b)
  real :: NestedDerivedTypeA(4)                ! GOOD
  real, allocatable :: NestedDerivedTypeB(:)   ! GOOD
  real :: NestedDerivedTypeA(b)                ! BAD
4. Crash pointing to problem with allocatable arrays which are used in OpenMP region. Message: "Subscript #x of the array has value xxxx which is greater than the upper bound of ..."
Reason: Known bug in Intel Fortran Compiler that occurs when code compiled using the /check:pointer option (under the Runtime category in project properties).


Derived Data Type - Nested
1. Complicated derived data types that involves nested derived types will not be able to be displayed in the debuger / variable watch space. The displayed numbers are grossly in error.


DLL not exposed properly -
When calling a function in a dll, but that function has not been exposed, then the following error may occur:
"The procedure entry point ..... could not be located in the dynamic link library ....dll"


VSL/MKL errors
Message:
MKL ERROR : Parameter 2 was incorrect on entry to vslNewStre
Cause:
using MKL, VSL, VML routines from intel, and having directives like:    
!dec$objcomment lib:'mkl_c_dll.lib'
    !dec$objcomment lib:'mkl_ia32.lib'
   are missing the path to the ...mkl\ia32\lib
Solution:
In VS.Net, within the dll/console project that uses them, add the path to the library files in:
Project -> Properties -> Linker -> General -> Additional Library Directories

IMSL Errors
Message:
Error: There is no matching specific subroutine for this generic subroutine call.
Cause:
   IMSL documentation shows Fortran90 version with D_RNCHI, but unless using somehow, still obeying
Fortran77. So use Fortran77 name which is DRNCHI.
Solution:
        Instead of using Fortran90 style -> D_RNCHI
we use -> DRNCHI

ThreadChecker Errors:
Problem Description:We recently received several problem reports. If the size of user's application is extreme big, the user complained that the application (launched by Thread Profiler) ran slowly.
Cause:Thread Profiler's engine uses 600MB (default) in the heap. If the application also needs to consume higher memory space in the heap and the user works on lower hardware (memory) configuration, it causes this problem
Resolution:  Use Configure -> Modify -> "Execution" tab -> "Limit the size of the heap of the heap used by the analysis engine to [ ] MB", adjust to smaller number. Note when Thread Checker reaches the memory limit, it may discard older statistics, causing some loss of results.



How to Add Version and other Metadata to DLL or EXE
=====================================================
Assume platform is Intel Fortran 8.1 and VS.Net 2003, but may apply to later versions too.
1. Go to Solutions Explorer and right click on the project name.
2. Choose Add New Item. In the Add New Item dialog, choose resource. A resourceX.rc will be created in the "Resource Files" folder directly under the project directory. Perhaps if this file already exist, we can skip to the next step.
3. Double click to open the resourceX.rc file.
4. In the resourceX.rc file, right click on the name resourceX.rc and choose "Add resource..."
5. In the "Add Resource" dialog, choose Version.
6. Fill in the relevant versioning and metadata info that is required.
7. Then build the project.
8. Check by right-clicking on the dll or exe file.


Using VTune
============
To use VTune, the following needs to be set up:
1. From VS.NET -> Project -> Properties -> Linker -> Debug -> Generate Program Database File
.... ensure this pdb file is defined.
From VS.NET -> Project -> Properties -> Fortran -> Debugging -> Debug Information Format
.... Full(/Zi)


2. Put this "/FIXED:NO" in:
VS.NET -> Project -> Properties -> Linker -> Command Line -> Additional Options
.... this is to ensure that VTune's Call Graph can be used. This only applies to the executable project.

3. Application to Launch - select and app or driver/dll that is already running.
Call Graph - must specify application to Launch.
Sampling and Counter may select "No App to launch"

4. Counter Monitor - Intel recommend using this first.
- uses native Windows performance counters, eg. processor queue, memory, processor time
- Has the following info:
   - the Logged Data view
   - the Legend
   - the Summary view
   - the Data Table - click on Logged Data View first to access
- Two main monitors to check are:
   - %Processor Time: The closer to 100% the better. This is calculated by taking amount
     of time spent in the Idle thread and subtracting from 100%
   - System Processor Queue length - There is a single queue for processor time even on
     multiprocessor systems. This counter should be less than 2. It measures how many
threads are waiting to execute.
- Intel Tuning Advice - to get the advice, from the Logged Data View, highlight the
  section of the graph of interest. Then click on the Tuning Assistant button.
- Drill Down to Correlated Sampling Data View.
   - To use sampling data, need to collect sampling data when collecting counter data.

5. Sampling Mode
- Look at Samples or Events of CPU_CLK_UNHALTED.CORE --- CPU cycles when a core is active
This shows where most cpu cycles are used.

     Definitions:
CPU_CLK_UNHALTED.CORE
Event Code: Counted by fixed counter number 1
Category: Basic Performance Tuning Events;Multi-Core Events;
Definition: Core cycles when core is not halted.
Description: This event counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. This event is a component in many key event ratios.
In mobile systems the core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. In systems with a constant core frequency, this event can give you a measurement of the elapsed time while the core was not in halt state by dividing the event count by the core frequency.

INST_RETIRED.ANY
Event Code: Counted by fixed counter number 0
Category: Basic Performance Tuning Events;
Definition: Instructions retired.
Description: This event counts the number of instructions that retire execution. For instructions that consist of multiple micro-ops, this event counts the retirement of the last micro-op of the instruction. The counter continues counting during hardware interrupts, traps, and inside interrupt handlers.

Clocks per Instructions Retired - CPI
Equation: CPU_CLK_UNHALTED.CORE / INST_RETIRED.ANY
Category: Basic Performance Tuning Ratios; Ratios for Tuning Assistant Advice;
Definition: High CPI indicates that instructions require more cycles to execute than they should. In this case there may be opportunities to modify your code to improve the efficiency with which instructions are executed within the processor. CPI can get as low as 0.25 cycles per instructions.

SAV = Sample After Value
This is the sampling frequency used for the sampling process. Typically it is 2,000,000.


Compiler Options
===================
/iface:[no]mixed_str_len_arg
Default: /iface:nomixed_str_len_arg

Specifies the type of argument-passing conventions used for general arguments and for hidden-length character arguments.
Possible values are:
/iface:mixed_str_len_arg: The hidden lengths should be placed immediately after their corresponding character argument in the argument list, which is the method used by Microsoft* Fortran PowerStation.
/iface:nomixed_str_len_arg: The hidden lengths should be placed in sequential order at the end of the argument list. When porting mixed-language programs that pass character arguments, either this option must be specified correctly or the order of hidden length arguments changed in the source code.

See also Programming with Mixed Languages Overview and related sections.

Compiling - Diagnostics.
To perform diagnostics such as using Vtune, Thread Profiler or Thread Checker, some of these options may be needed:
/Zi - include symbols   = /debug:full
/Od - disable debugging  
/fixed:no - linked to make code relocatable
/MDd - to build with thread safe libraries =   /libs:dll /threads /dbglibs




Build Macros (eg $(OUTDIR))
============================
See Intel Visual Fortran - User Guide - Volume I: - Building apps from MS Visual Studio.Net - Supported Build Macros.

Example:
   In the project properties - Linker - Output File, the value is "$(OUTDIR)/xxx.dll".
   The macro $(OUTDIR) has a value defined in:
       project properties - Output Directory
   Similarly $(INTDIR) is defined in
       project properties - Intermediate Directory

Using MKL
===========
1. CBA desktop PC - Pentium 4 CPU 3.8GHz
   - from intel website:
       CPU No.: 670;   90 nm;   Cache: 2 MB L2;   Clock Speed: 3.80 GHz;  FSB: 800 MHz
  Hyperthreading, Enhanced SpeedStep, Intel64 (need Bios and OS), ExecuteBit Enabled

2. Installation Directories:
   - c:\Program Files\intel\mkl\8.1.1
   - tools\environment -> mklvarsXXX.bat to build environment variables.
   - 3 options: ia32, em64t, ia64; within these are dlls and libs files
   - for the ia32 option, ia32\bin contain:
mkl_lapack_YY.dll, mkl_XXX.dll, mkl_vml_XXX.dll, mkl_ias.dll, libguide40.dll    
YY = 32,664
XXX = def, p3,p4, p4p, p4m
   - for the ia32 option, ia32\lib contain:
mkl_X.lib, mkl_X_dll.lib, mkl_lapack.lib, mkl_solver.dll, mkl_ia32.lib, libguide40.lib, libguide.lib
X = c (for c), s (for Fortran)

3. Configuring to use MKL
- at installation time, say yes to add vars to PATH, LIB, INCLUDE.
- alternatively, run mklvars32.bat

4. Using Fortran95 BLAS or LAPACK
    - Need to build from Intels sources, go to mkl\8.1.1\interfaces\blas95,lapack95
- nmake PLAT=win32 lib -> a *.mod file will be created
- or go to INCLUDE directory and: ifort -c mkl_lapack|blas.f90
- Or to make it in the user's directory:
 1. copy mkl\8.1.1\interfaces\blas95,lapack95 into
 2. copy from INCLUDE to these files: mkl_lapack|blas.f90
 3. run in the blas,lapack directories: nmake PLAT=win32 INTERFACE=mkl_blas|lapack.f90 lib
for 64 bit
    - nmake can be found in C:\Program Files\Microsoft Visual Studio 8\VC\bin\
- from the Start Menu, open Intel Visual Fortran Build Environment using Intel 64.
- nmake PLAT=win32e lib
- mod files will be automatically copied to ..../em64t


5. Linking to library:
a) see "Linking your application with Intel MKL" in "Getting Started with the Intel Math
Kernel Library 8.1.1 for Windows" for reference.
b) In VS.Net, go to Project menu -> Properties -> Linker -> General -> Additional Library Directories
   and put:
C:\Program Files\Intel\MKL\8.1.1\ia32\lib

6. Errors
a) Compile error:
SortProj1  error LNK2019: unresolved external symbol _VSLNEWSTREAM referenced in function _MAIN__.L
   Solution:
   1. put the following in the code at start of module or program, NOT subroutine or function
    use MKL_VSL_TYPE
    use MKL_VSL
    !dec$objcomment lib:'mkl_c_dll.lib'
    !dec$objcomment lib:'mkl_ia32.lib'
   2. Could also be sometimes need DLLIMPORT rather than DLLEXPORT, especially in RELEASE version????
   3. If the function is a Fortran95 function, such as gemv, then the solution is to "call dgemv.." rather
      than "call gemv..."

b) Runtime error:
MKL ERROR: Parameter 2 was incorrect on entry to vslNewStre
   Solution:
   In VS.Net, go to Project menu -> Properties -> Linker -> General -> Additional Library Directories
   and put:
C:\Program Files\Intel\MKL\8.1.1\ia32\lib

7. Prerequisite Directories - these need to be put in Project -> Properties or command line or etc...
  1. Include Directories: C:\Program Files\Intel\MKL\8.1.1\include
  2. Library Directories: C:\Program Files\Intel\MKL\8.1.1\ia32\lib
  3. Put the following line in the start of one of the source code, before the program or module keyword.
 include 'mkl_vsl.fi'    ! This is a full-fledged module by MKL
  4. Put the following at the start of a module or program, not within a function or subroutine
    use MKL_VSL_TYPE
    use MKL_VSL
    !dec$objcomment lib:'mkl_c_dll.lib'
    !dec$objcomment lib:'mkl_ia32.lib'    implicit none


Using LAPACK95 & General Comment on DLLs, LIBs, Mod files
==========================================================
   To illustrate the usage of Lapack functions with Fortran95 interface,
 suppose we want to use subroutine GESV
Fortran77 call: sgesv, dgesv, cgesv, zgesv
Fortran95 call: gesv

gesv is an Interface in mkl_lapack.f90(module MKL95_LAPACK)
gesv interface overloads wrappers like DGESV_MKL95, etc....

Only two items are needed by the user -> *.lib and *.mod

DLL
- not needed because we will be using explicit interfaces.
- Also F95 lapack routines have optional arguments which REQUIRE interfaces (eg gesv).

LIB
- mkl_lapack95.lib needed (created once off by administrator or first user)
- Use in the code as:
!dec$objcomment lib:'mkl_lapack95.lib'
    !dec$objcomment lib:'mkl_c_dll.lib'
    !dec$objcomment lib:'mkl_ia32.lib'  
- Don't need
!dec$objcomment lib:'mkl_lapack.lib'
- must be linked during compile time either
    i) ifort ..... mkl_lapack95.lib; or
ii) specify the path in "Additional Library Directories"

MOD
- mkl95_lapack.mod needed (created once off by administrator or first user from mkl_lapack.f90)
- contains the collection of interfaces to be used in the code by having:
USE MKL95_LAPACK
- must be present during compile time in the directory path of either:
    i) same location as application source files.f90
ii) INCLUDE directories as specified in VS.Net as "Additional Include Directories"


Mixed language programming
============================
Hi Clinton,
It looks like library format incompatibility problem. We adhere to microsft format.
Please follow following steps as a work-around ;
Once you generate .dll from intel FORTRAN compiler; follow the following steps,

1. D:\>pedump /exp MatlabFunctions.dll > MatlabFunctions.exp

D:\>notepad MatlabFunctions.exp (Edit this file and replace MATEXP with _MATEXP)

D:\>buildlib MatlabFunctions.exp MatlabFunctions.lib

D:\> lcc hello.c

D:\>lcclnk hello.obj MatlabFunctions.lib

D:\>hello.exe

Stack Checking
===============
Checking and Setting Space
The following options perform checking and setting space for stacks (these options are supported on Windows only):

The /Gs0 option enables stack-checking for all functions.
The /Gsn option checks by default the stack space allocated for functions with more than 4KB.
The /Fn option sets the stack reserve amount for the program. The /Fn  option passes /stack:n to the linker.



Enable Vectorization and Report
================================
To enable automatic vectorization, use these switches:
   /Qx...  or /Qax....
To enable report, use:
   /Qvec-report....


Enable OpenMP
==============
1. To enable openMP;
  by Command line: /Qopenmp /Qfpp
  by VS.net:  Project -> Properties -> Preprocessor -> OpenMP conditional compilation -> Yes
              Project -> Properties -> Preprocessor -> Preprocess source file -> Yes (/fpp)
              Project -> Properties -> Language -> Process OpenMP directives -> Generate Parallel code (/Qopenmp)

Note: preprocessor must be enabled for the OpenMP directives to be processed.

2. For diagnostic report:
   by Command line: /Qopenmp-report

3. Compile OpenMP but in sequential mode;
   by Command line: /Qopenmp-stubs

or to Compile for single thread, use the preprocessor /Qfpp, but not the OpenMP /Qopenmp.

4. DO NOT USE /Qprof-genx with OpenMP - spurious errors like array out of bounds will result.

5. To use OpenMP functions like, omp_get_num_threads(), instead of using
     include "omp_lib.h",
   better to use:
        external omp_get_num_threads
        integer omp_get_num_threads


Using Thread Profiler
=======================
1. Compiler options to enable Thread Profiling:
a) /Zi         - full debugging format
b) /fixed:no   - linker option to make code relocatable
c) /MDd        - option tells the linker to search for unresolved references
               in a multithreaded, debug, dynamic-link (DLL) run-time library.
               This is the same as specifying options /libs:dll /threads /dbglibs.
d) /Qopenmp-profile - enable profiling of OpenMP.
   WARNING: this option should not be used with IMSL since IMSL will link to libguide or libguide40, but
   this option creates code that will link to libguide_stats or libguide40_stats


Using Thread Checker
=====================

Add the following library path:
     .....VTune\Analyzer\Lib
without this compiling error occurs stating that libassuret40.lib was not found.

Options for Thread Checker
/ZI, /Z7 (Fortran - General - Debug Information Format - Full)
/Od (Fortran - Optimization - Optimization - Disable)
/libs:dll /threads /dbglibs (Fortran - Libraries - Runtime Library - debug Multithreaded Dll)
/Qtcheck - to enable use by Thread Checker

To run Intel Thread Checker, run VTune first in a NEW project. When the VTune is finished analysing, then run thread checker from the SAME project, by running as a NEW Activity.

Troubleshooting:
- ensure /Qtcheck is only on the EXECUTABLE, not other dlls.
- check working directory is correct.
- when EXECUTABLE has /Qtcheck, it cannot be run from console mode.


Profile Guided Optimization
============================
This is a 3 step process:
1. Compile with /Qprof-gen. Using /Qprof-genx allows Code Coverage tool to be used.
   DO NOT USE /Qprof-genx WITH OPENMP.
   Note: For Code Coverage, new option is /Qprof-gen:srcpos

2. Run the code one or many times with different data sets.
   This will create .dyn files.
3. Compile with /Qprof-use. This uses the .dyn file created in step 2.
4. Usually specify /O2 in step 1, and more aggresive /Qipo in step 3.
5. Need the following:
C:\Program Files\Microsoft Visual Studio 8\Common7\IDE;
C:\Program Files\Microsoft Visual Studio 8\VC\BIN;
C:\Program Files\Microsoft Visual Studio 8\Common7\Tools;
C:\Program Files\Microsoft Visual Studio 8\Common7\Tools\bin;
C:\Program Files\Microsoft Visual Studio 8\VC\PlatformSDK\bin;
msvcr80d.dll -> C:\Program Files\Microsoft Visual Studio 8\VC\redist\Debug_NonRedist\x86\Microsoft.VC80.DebugCRT


Using code coverage
====================
Ref: Intel_compiler_code-coverage.pdf

To use code coverage which is available for Intel compilers, the code needs to be prepared during compilation, then the application need to be run. The following is the general method.

1. Compile source code with /Qprof-gen:srcpos option.
By default, pgopti.spi, a static profile file is created. This name can be changed using the -prof-file option.
2. Run the application. This will create multiple dyn files for dynamic profile information.
3. Use the profmerge tool to merge dyn files into pgopti.dpi file.
     profmerge -prof_dpi
4. Run code coverage using both static and dynamic files
     codecov -spi -dpi
5. The results are published into CODE_COVERAGE.HTML

Note that these commands should run in the same directory as the source code and execution directory.



NOTES IIS web server


IIS - Internet Information Services

Contents
=========
Install
Configuration
Start-Stop
Tutorials
Test IIS Working



Install
========
1. Installed as part of operating system. Go to:
Control Panel -> Add / Remove Programs -> Add Windows components -> IIS

2. Versions - IIS version is tied to the Operating System.
IIS 5 - is associated with Windows 2000 (all versions)
IIS 5.1 - is associated with Windows XP Professional
IIS 6 - is associated with Windows .NET Server
IIS versions 3 and 4 are designed for Windows NT 4.0,
(technical support for this is expected to be terminated by the end of 2002)[1].

3. Security Issues (Hardening)
Ref: http://www.windowsecurity.com/articles/Installing_Securing_IIS_Servers_Part1.html

4. On a Command Console:
- goto c:\Windows\Microsoft.Net\Framework\vx.xxxx
- type: aspnet_regiis -i


Configuration
==============
1. To open the configuration box:
i) Control Panel -> Administrative services -> Internet Information Services
ii) c:\windows\system32\inetsrv\iis.msc

2. For each ASP.NET web application, ensure folders have Read and Execute permissions

Start-Stop
===========
To control the start/stop/automatic-startup ->
Control panel -> administrative tools -> Component Services -> services -> IIS Admin


Tutorials
==========
IIS online getting started: http://localhost/iishelp/iis/misc/default.asp

Test IIS Working
=================
To test that IIS is working with asp pages:
1. Open browser.
2. Browse to URL: http://localhost/localstart.asp
3. If the Welcome to IIS page is not displayed, then check that IIS is working.
4. IIS control panel: system32\inetsrv\iis.msc
5. Under IIS -> ComputerName -> Websites: Check that "Default Website" is "running" state.

Notes Ant


Introduction
Install
Running Ant
Structure
Example
Using Ant with Eclipse


Introduction
============
- Ant is like a Makefile. The configuration is called by default as
"build.xml"
- Ant is a Java program.
- Requires xml parsers and Java installed


Reference: http://ant.apache.org/manual/index.html



Install
==========
1. Unpack ANT into a directory eg. c:\ant.
2. Set Env variables:

Windows and OS/2
Assume Ant is installed in c:\ant\. The following sets up the environment:

set ANT_HOME=c:\ant
set JAVA_HOME=c:\jdk1.2.2
set PATH=%PATH%;%ANT_HOME%\bin

(Start - > Control Panal -> System -> Advanced ->Env Variables)

Unix (bash)
Assume Ant is installed in /usr/local/ant. The following sets up the environment:

export ANT_HOME=/usr/local/ant
export JAVA_HOME=/usr/local/jdk-1.2.2
export PATH=${PATH}:${ANT_HOME}/bin

(/etc/profile.d/*.sh)


Running Ant
============
To run:
ant -buildfile

ant -buildfile test.xml -Dbuild=build/classes dist
runs Ant using the test.xml file in the current directory,
on the target called dist, setting the build property to
the value build/classes.


Structure
==========
Structure of build.xml:
attribs: name, default (target), basedir
Children: , ,

attribs: name, depends, if, unless, description
Children: various tasks

Tasks
attribs: id

attribs: name, value, file, location, url, refid, resource,
environment, classpath, classpathref, prefix
Example
========
simple example build file



The following allows ant to make use of local environment variables







Using Ant with Eclipse
=======================
1. Create a new xml in Eclipse, to be the ant build file, eg. build.xml
2. Open the build.xml file with Eclipse editor
3. Edit the xml file, note there is context sensitive help in Eclipse.
4. To run the build.xml: Run As -> Ant Build, then select targets
to be build.

Notes 64bit

Notes64bit

Contents
============
References
Definition
Article - The 64-Bit Advantage
Article - x86: registered offender
Itanium2
Feature Comparison
Registers
AMD K8 vs Conroe FPU
Porting to a 64-bit Intel® architecture
How to check if code is 32bit or 64bit
Limitations
Large Arrays



References
=============
http://www.intel.com/cd/ids/developer/asmo-na/eng/197664.htm?page=5


Definition
===========
Ref: http://en.wikipedia.org/wiki/64-bit
"64-bit" computer architecture generally has integer registers that are 64 bits wide, which allows it to support (both internally and externally) 64-bit "chunks" of integer data.



Size to consider are: registers, address buses, or data buses.


Most modern CPUs such as the Pentium and PowerPC have 128-bit vector registers used to store several smaller numbers, such as 4 32-bit floating-point numbers. A single instruction can operate on all these values in parallel (SIMD). They are 128-bit processors in the sense that they have 128-bit registers and in some cases a 128-bit ALU, but they do not operate on individual numbers that are 128 binary digits in length.


Article - The 64-Bit Advantage
===============================
Ref: http://www.pcmag.com/print_article/0,3048,a=116259,00.asp

The 32-bit Pentium-class chips that dominate today's desktops fetch and execute instructions from system memory in 32-bit chunks; 64-bit chips handle 64-bit instructions. And that's just what the workstation-class Intel Itanium 2 and HP Alpha chips do inside the TeraGrid's clusters.

New desktop-class 64-bit chips, such as the AMD Athlon64 and the Apple/ IBM PowerPC G5, can handle 64-bit instructions as well, but most PC apps—even the few that optimize some operations to exploit 64-bit processing—still rely on 32-bit instructions. A new generation of games and apps will no doubt take fuller advantage of 64-bit chips. But their ability to harness the new architecture fully may be hampered by the need to interact with Windows, since none of the desktop versions of the OS is yet slated for 64-bit optimization.

A major advantage to 64-bit processors over their 32-bit cousins is support for greater amounts of memory. In theory, a 64-bit processor can address exabytes (billions of billions of bytes) of RAM; 32-bit chips can use a maximum of 8GB of RAM. This breakthrough is used to good advantage at the National Center for Supercomputing Applications' (NCSA) TeraGrid, which allocates 12GB of system memory each to half of its 256 Itanium 2 processor nodes. It will be a while before anyone knows how fast Quake would run with that much memory, since PC motherboards don't exceed 8GB of RAM.

Future 64-bit apps will be able to chew on a class of computations known as floating-point operations far faster than 32-bit apps can. Necessary for 3-D rendering and animation of everything from molecular models to Halo aliens, floating-point calculations are so essential to complex scientific analysis that FLOPS (floating-point operations per second) are used as the unit of supercomputing performance. The ability of 64-bit chips to process floating-point operations faster and far more precisely than their 32-bit counterparts make them powerhouses for simulations and visualization.


Article - x86: registered offender
===================================
Ref: http://techreport.com/reviews/2005q1/64-bits/index.x?pg=2

"Another problem with the x86 ISA is the number of general-purpose registers (GPRs) available. Registers are fast, local slots inside a processor where programs can store values. Data stored in registers is quickly accessible for reuse, and registers are even faster than on-chip cache. The x86 ISA only provides eight general-purpose registers, and thus is generally considered register-poor. Most reasonably contemporary ISAs offer more. The PowerPC 604 RISC architecture, to give one example, has 32 general-purpose registers. Without a sufficient number of registers for the task at hand, x86 compilers must sometimes direct programs to spend time shuffling data around in order to make the right data available for an operation. This creates overhead that slows down computation.

To help alleviate this bottleneck, the x86-64 ISA brings more and better registers to the table. x86-64 packs 8 more general-purpose registers, for a total of 16, and they are no longer limited to 32-bit values—all 16 can store 64-bit datatypes. In addition to the new GPRs, x86-64 also includes 8 new 128-bit SSE/SSE2 registers, for a total of 16 of those. These additional registers bring x86 processors up to snuff with the competition, and they will quite likely bring the largest performance gains of any aspect of the move to the x86-64 ISA.

What is the magnitude of those performance gains? Well, it depends. Some tasks aren't constrained by the number of registers available now, while others will benefit greatly when recompiled for x86-64 because the compiler will have more slots for local data storage. The amount of "register pressure" presented by a program depends on its nature, as this paper on 64-bit technical computing with Fortran explains:

The performance gains from having 16 GPRs available will vary depending on the complexity of your code. Compute-intensive applications with deeply nested loops, as in most Fortran codes, will experience higher levels of register pressure than simpler algorithms that follow a mostly linear execution path. "

Summary -
x86 - 8x 32-bit General Purpose Registers
x86-64 - 16x 64-bit General Purpose Registers
Fortran - more do loops need bigger and more GPRs


Itanium2
=========
Ref: http://www.itmanagersjournal.com/feature/8611

Intel's Itanium 2, or IA64, is unlike any of the other 64-bit processors in production. It uses a Very Long Instruction Word (VLIW) design that depends on the software's compiler for performance. When the compiler creates program binaries for the Itanium 2, it predicts the most efficient method of execution, so the processor does less work when the program is running -- the software schedules its own resources beforehand, rather than forcing the hardware to do it on the fly. IA64 is used in the same kinds of workstations that UltraSPARC processors are used in, and can also scale up to 128 processors in high-powered servers. Silicon Graphics and Hewlett-Packard both sell computers based on the Itanium 2. GNU/Linux is generally the operating system of choice for IA64-based systems, but HP-UX and Windows 2003 Server will work on HP Itanium 2 servers.



Feature Comparison
==================

Size of Fetch and Execute Instructions - 64bit vs 32bit chunks
Number of General Purpose Registers (Fetch Registers?)
Memory Access - 18.4x10^9 GB vs 4GB
Floating Point Operations - faster with 64bit than 32bit

Vector Registers - eg Pentium has 128bit data registers which store up to 4 32bit data register.
ALU
FPU


Itanium2
- good for FP processing
- 2FPU (=1 FMAC or 2multiplication and 1 add), plus additional 2 FMACs for 3D processing.
- 64 bit address space
- a derivative of VLIW, dubbed Explicitly Parallel Instruction Computing (EPIC). It is theoretically capable of performing roughly 8 times more work per clock cycle than a non-superscalar CISC or RISC architecture due to its Parallel Computing Microarchitecture.
- support 128 integer, 128 floating point, 8 branch and 64 predicate registers (for comparison, IA-32 processors support 8 registers and other RISC processors support 32 registers


UltraSparc T1
- 8x Integer Cores share 1 FPU
- good for integer processing compared to Itanium


Throughout its history, Itanium has had the best floating point performance relative to fixed-point performance of any general-purpose microprocessor. This capability is not needed for most enterprise server workloads. Sun's latest server-class microprocessor, the UltraSPARC T1 acknowledges this explicitly, with performance dramatically skewed toward the improvement of integer processing at the expense of floating point performance (eight integer cores share a single FPU). Thus Itanium and Sun appear to be addressing separate subsets of the market. By contrast, IBM's cell microprocessor, with a single general-purpose POWER core controlling eight simpler cores optimized for floating point, may eventually compete against Itanium for floating-point workloads.


Registers
==========
Integer - can be used to store pointers
Floating Point - most CPUs also have FPUs
Other

examples:
x86 - has x87 FPU with 8 x 80bit registers
x86 with SSE - 8x 128bit FP registers
x86-64 - has SSE with 16x 128bit FP registers
Alpha - has 32x 64bit FP registers and 32x 64bit integer registers.
Itanium2 - 128x 64bit GPRs, 128x 82bit FPregisters, 64x 1bit predicates, 8x 64bit branch registers

AMD K8 vs Conroe FPU
======================
Possibly better floating point performance of K8 processors

http://www.xbitlabs.com/articles/cpu/display/amd-k8l_5.html


Porting to a 64-bit Intel® architecture
============================================
(ref: http://www.developers.net/intelisnshowcase/view/358)

Porting Application Source Code
The most significant issues that software developers should face in porting source code to the 64-bit world concern the changes in pointer size and fundamental integer types. As such, these differences should appear most prominently in C and C++ programs. Code written in Fortran, COBOL, Visual Basic, and most other languages (except assembly language, which must be completely rewritten), will need no modification. A simple recompilation is often all that is needed. Java code should not even need recompilation; Java classes should execute the same on a 64-bit JVM as on any 32-bit virtual machine.

C (from here on, C++ is included in all discussions of C) code, however, by allowing casting across types and direct access to machine-specific integral types will need some attention.

The first aspect is the size of pointers; 64-bit operating systems use 64-bit pointers. This means that the following will equal eight (and no longer four):

sizeof (ptrdiff_t)

As a result, structures that contain pointers will have different sizes as well. As such, if data laid out for these structures is stored on disk, reading it in or writing it out will cause errors. Likewise, unions with pointer fields will have different sizes and can cause unpredictable results.

The greatest effect, though, is felt wherever pointers are cast to integral types. This practice, which has been condemned for years as inimical to portability, will come back to haunt programmers who did not abandon it. The problems caused by it are traceable to the different widths used by pointers, integers and longs on the various platforms. Let's examine these.


How to check if code is 32bit or 64bit
========================================
use the dumpbin utility and look for the output under FILE HEADER VALUES.
eg.
dumpbin /headers hello.exe

Results if 64 bit:
FILE HEADER VALUES
8664 machine (x64)

Results if 32 bit:
FILE HEADER VALUES
14C machine (x86)

Limitations
=============
Virtual Address Limit - theoretical 16EB
Virtual Address Limit - practical
i) Windows use 44bits -> 16TB, apparently allow only 8TB to be used.


Large Arrays
=============
http://episteme.arstechnica.com/eve/forums/a/tpc/f/6330927813/m/420003239831/r/308002539831

BigArray, getting around the 2GB array size limit
http://blogs.msdn.com/joshwil/archive/2005/08/10/450202.aspx

Monday, September 07, 2009

Branching and Merging using TFS

The concept of branching and merging is simple but the mechanics of using TFS for branching and merging need to be understood carefully because it is prone to errors when users misses a step. Quick advice, practice branching and merging a few times with test projects before using it on your real project.

This section will outline a step by step walkthrough of the Branch-Merge process using TFS. But before the walkthrough, please go through the pre-requisites here and be thoroughly familiar before moving on.

Pre-requisites:
1. Read the official documentataion (this is not enough, but still need to be familiar with) at:
MSDN Library - Development Tools and Languages - Visual Studio Team System - Team Foundation - Team Foundation Project Members -
Working with Team Foundation Source Control - Branching and Merging Team Foundation Source Control.

2. Be familiar with Solution Explorer in Visual Studio (VS) - if not shown then access from View menu. Understand that Solution Explorer shows the LOCAL version of your code.

3. Know how to check-out and check-in projects between Local workspace and TFS server. This is another big area which you need to fully understand.

4. Be familiar with Source Control Explorer. This is a client to view your files that are stored in the TFS server. This is a server view of files on the server - they are not your local files.

5. Change Source Control. This is a crucial tool to check that your projects are in sync between LOCAL versions and TFS server versions. Access this by highlighting a project in Solution Explorer, then go to File - Source Control - Change Source Control.
Note that if the project is not in source control, then the link above will be: File - Source Control - Workspaces.

6. There is an existing project (create one if needed) called Test_BranchMerge in a Solution of your choice. This Solution and its project should be in your local PC and in the TFS server and is properly checked-in and the source control path properly synchronized (can be checked using Change Source Control - see step 5 above). In this project, there should be at least one source file, eg Source1.f90. This example project is a Fortran project, but it should apply to other projects recognized by VS.


Walkthrough of Branching and Merging:
1. Go to Solution Explorer and click on the Test_BranchMerge project. Check the following: i) On the Pending Change column, there is nothing pending. ii) in the Latest column, everything is Yes. iii) In the Local Path above, the path is pointing to the correct local folder.

2. BRANCH - In the Solution Explorer, right click on Test_BranchMerge project; select Branch. In the Branch dialog:
i) in the Target field, you change the last part of the name, but do not alter the other parts of the path. Example:
Target: $/aaa/bbb/ccc/ddd/Test_BranchMerge-branch
this can be changed to
Target: $/aaa/bbb/ccc/ddd/Test_BranchMerge-06
but do not change the server structure $/aaa/bbb/ccc/ddd
ii) Branch from Version: select Latest Version.
iii) Check the box "Create local working copies for the new branch.
iv) Click OK.
This process will create a branch in the TFS server as well as a copy of the project in your local directory under the same solution as the original project source.

3. VERIFY -
i) In Windows File Explorer (not in VS), the folder Test_BranchMerge-06 exist on the same level as Test_BranchMerge. All contents between these two folders should be identical.
For Advanced Users: the project files *.cproj (for C#), *.vfproj (for Fortran) which identifies the projects to VS are identical between Test_BranchMerge-06 and Test_BranchMerge. This means all systems including VS (except for TFS) will REGARD these two projects as IDENTICAL. In VS, Test_BranchMerge-06 will be recognized as Test_BranchMerge. This is the correct feature and effect we need for branching. See step 4 in using this new branch to work with.
ii) In the VS - Source Control Explorer, Test_BranchMerge-06 is visible on the Directory Tree on the same level as Test_BranchMerge, with a branching symbol next to it. Click on Test_BranchMerge-06 on the left pane and on the right pane, all the contents Test_BranchMerge-06 are visible. All contents have the branching symbol next to it. In the Pending Change column, all contents have state - branch.
iii) In the Solution Explorer, check that the Test_BranchMerge project is still visible. Note that this shows the local Solution which is still linked to the local source of the original Test_BranchMerge project. This will be re-connected manually in step 4. Click to highlight Test_BranchMerge project in Solution Explorer, then go to File - Source Control - Change Source Control. View that the Test_BranchMerge project has Server Binding pointing to $/aaa/bbb/ccc/ddd/Test_BranchMerge. Do not change anything. Click OK.
iv) Again highlight Test_BranchMerge in Solution Explorer, then look at the Properties Window (if not show, go to View menu).
Verify that the project path is still pointing to Test_BranchMerge folder in your local PC.

4. INITIAL CHECK-IN
This step is necessary if we want to work on the changes in the branched code. At the end of Step 2, although the branched project and source has been created in TFS and the local PC, it HAS NOT BEEN REGISTERED UNDER TFS.
i)In the Source Control Explorer, in the left pane, right click on Test_BranchMerge-06 and select: Checkin Pending changes.
ii) Verify - Look in the right pane, under the Pending Changes column and check that all the files under Test_BranchMerge-06 has NO pending status at all; ie. the branching symbols and branch pending status are gone.
At this stage, the new branch for the project has been cloned.


5. RE-CONNECT VS Solution to point to the new Branch.
At this stage, VS Solution is still connected to Test_BranchMerge folder locally.
i) Go to Solution Explorer, right click on Test_BranchMerge and select Remove.
ii) In the Solution Explorer, right click on the Solution (at the top) and select Add - Existing Project.
iii) In the dialog box that appears, navigate to Test_BranchMerge-06 folder on your local PC and select Test_BranchMerge.vfproj or Test_BranchMerge.csproj, etc... and click Open. Note that the project file is STILL named Test_BranchMerge.xxxx, not Test_BranchMerge-06.xxxx.

6. VERIFY - that the new branch is connected.
i) From the Solution Explorer, highlight the project Test_BranchMerge. In the Properties Window, check that the Project Path is pointing to the local folder Test_BranchMerge-06.
ii) From the Solution Explorer, highlight the project Test_BranchMerge. Go to File - Source Control - Change Source Control. Looking at the project Test_BranchMerge - check that its Server Binding is set to $/aaa/bbb/ccc/ddd/Test_BranchMerge-06. Check that the Connected box is ticked and the Status is Valid.
This shows that for all intent and purposes, the branch is now recognized as the Test_BranchMerge project (not Test_BranchMerge-06). The paths to the server and local PC shows that the project is linked to the correct branch folder Test_BranchMerge-06.

7. WORK IN THE BRANCH
In the VS environment, modify the code, run the program, do whatever as normal. The Test_BranchMerge project should work as usual before branching. Even the Check-in proces to TFS should work as normal. As noted before, the source code is now connected to the branched version Test_BranchMerge-06, both locally and on TFS.
i). In the Solution Explorer, in the Test_BranchMerge project, open up the file Source1.f90. Edit this file, save it.
ii). Note the TFS locked symbol next to Source1.f90 changed to the red tick symbol indicating that the code is updated and different to the TFS version.
iii). If the red-tick edit symbol does not appear, then in Solution Explorer, click on Source1.f90, select Check-out for Edit. Then edit the code and save.
iv). Once all code changes are done, click on the Test_BranchMerge project in Solution Explorer and select Check In.

8. MERGE - like branching, merging must be done via Source Control Explorer. Branch - Merge are TFS operations, they are not VS operations.
i) From the Source Control Explorer, right click on the Test_BranchMerge-06 project, and select Merge.
ii) In the Merge dialog, choose the following:
Source Branch: $/aaa/bbb/ccc/ddd/Test_BranchMerge-06
Select: All changes up to a specific version
Target Branch: $/aaa/bbb/ccc/ddd/Test_BranchMerge
iii) Click Next, choose Latest Version as the Version Type, click Next, click Finish.
Note the merging occurs between code in TFS server, not your local version. Hence before merging, both old and new branch must be fully checked in. Also note merging occurs from Branched version as Source to Target original version.
iv) Verify - in the Source Control Explorer, click to highlight the original project Test_BranchMerge. Looking at the right pane under the Pending Change column, note that the files which has been changed will now have the "merge, edit" status.
v) Checkin - in the Source Control Explorer, right click Test_BranchMerge and select Checkin Pending Changes.
vi) Verify - if checkin is successful, then the files with "Merge, Edit" Pending Changes (step iv) above) will now have NO pending change status.

9. CONTINUING WORK
In VS, remember that the Test_BranchMerge project is still linked to the branched version Test_BranchMerge-06 code base. The user now has many options such as:
i) Continue working with the Test_BranchMerge-06 branch. The user can keep on editing code in VS under the Test_BranchMerge project, with no extra re-configuration.
ii) Revert back to work with original branch (which now has the merged changes). In the Solution Explorer, remove the Test_BranchMerge project (which is actually the Test_BranchMerge-06 branch). Then click on Solution in the Solution Explorer and Add Existing Project; select the xxxx.xxproj file from Test_BranchMerge folder.
iii) Work from a totally new branch, eg using a new branch Test_BranchMerge-07. To do this, repeat the whole process from Step 1.



There may be variations of how the branch-merge process can be done; the walk-through shown above is just one scenario that should just work without having to spend more time investigating an error prone process.

Thursday, September 03, 2009

NotesMSBuild

NotesMSBuild
=============

Contents
=========
Paths
MSBuild schema
MSBuild summary
MSBuild complete structure
Output Element
MSBuild tasks
MSBuild conditions
MSBuild reserved properties
MSBuild command line
MSBuild Well-known item metadata


Paths
======
When MSBUILD executes, the default root path of any relative path references made in the *.proj file, is the path where the *.proj file is executing from.
For example, if *.proj is in C:\test, then any relative path reference in the *.proj file starts from C:\test.


MSBuild schema
===============
The official schema or xsd for MSBuild can be found in:
Program Files\Microsoft Visual Studio 8\Xml\Schemas\1033\Microsoft.Build.xsd


MSBuild summary
================
Reference: In the MSDN, look for .Net Development - .Net Framework SDK - General Reference - MSBuild Reference.

MSBuild is a tool that lets developer perform various operations on a set of files given some specified conditions. To those who know the Makefile utility in Linux/Unix, MSBuild performs similar build tasks. From what I've read, MSBuild should be quite similar to NUnit and similar tools.

MSBuild is defined using XML file format, its structure is shown in the section "MSBuild complete structure" below.
Being XML based, it has the typical child elements and attributes structure that are in other XML formats.
The key items that drive MSBuild as I see it are the:
i) Targets - this allow the user to specify what to do
ii) Tasks - this allow the MSBuild designer to pre-specify the various tasks to be performed for each of the defined Targets.

The MSBuild design is typical saved as fileName.proj

To make use of the names defined in the MSBuild project file:
$()  Extracts the value of a property
@()  Extracts the value of an item as a list, that is, vector
%()  Extracts value of an item as a single string, that is, scalar


Example:

MSBuild complete structure
===========================
The complete MSBuild structure is shown in the tree / hierarchy below. The purpose of this diagram is to provide a quick illustration of the child elements and attribute relationships which is not easy to see from the official schema / xsd.

The top level element is the Project element.
: denotes attributes of the element
+ denotes child elements
Element names which begin with small letters, {item, itemMetaData, property, parameter} are actually
user defined names.



Project
    : DefaultTargets  eg DefaultTargets="tarA;TarB"
    : InitialTargets  eg InitialTargets="tarC;tarD"
    : xmlns           eg xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
    + Choose
        + Otherwise
            + Choose
            + ItemGroup
            + PropertyGroup
        + When
            : Condition,  eg Condition="'StringA'=='StringB'"
            + Choose
            + ItemGroup
            + PropertyGroup
    + Import
        : Condition
        : Project,  eg Project="ProjectPath"
    + ItemGroup
        : Condition
        + item
            : Condition
            : Exclude,  eg Exclude="MyFile.cs"
            : Include,  eg Include="*.cs"
            + itemMetaData
    + ProjectExtensions
    + PropertyGroup
        : Condition
        + property
            :Condition
    + Target
        : Condition
        : DependsOnTargets
        : Inputs
        : Name
        : Outputs
        + Error
            : Condition
            : Text
        + Message
            : Condition
            : Text
        + OnError
            : Condition
            : ExecuteTargets, eg ExecuteTargets="TargetName"
        + task
            : Condition
            : ContinueOnError   eg ContinueOnError="true/false"
            : parameter         eg par1="val1" par2="val2" par3="val3"
            + Output
                : Condition
                : ItemName      either ItemName or PropertyName is required
                : PropertyName
                : TaskParameter
        + Warning
            : Condition
            : Text
    + UsingTask
        : AssemblyFile
        : AssemblyName
        : Condition
        : TaskName














Output Element
==============
Output is an element not a task. The general structure for the Output Element is:

    [PropertyName="PropertyName" | ItemName="Item name"]
    Condition = "'String A' == 'String B'" />

Either the PropertyName or the ItemName attribute must be included with each Output element.
The value for the Output's TaskParameter must be the special name of the parameter of the task.
In the example below where Output is used with the Copy Task, the parameter "CopiedFiles" is a special name and is one of the defined parameters for COPY (see Copy task reference). The list of copied files is being held it the item called "SuccessfullyCopiedFiles". This list can be used by calling @(SuccessfullyCopiedFiles)



    
        SourceFiles="@(MySourceFiles)"
DestinationFiles="............"
        DestinationFolder="@(MyDestFolder)">
        
            TaskParameter="CopiedFiles"
            ItemName="SuccessfullyCopiedFiles"/>
    


MSBuild Tasks
==============
The following tasks list is taken from http://msdn.microsoft.com/en-us/library/7z253716(VS.80).aspx


AL (Assembly Linker) Task
Describes the AL task and its parameters.

AspNetCompiler Task
Wraps aspnet_compiler.exe, a utility to precompile ASP.NET applications.

AssignCulture Task
Assigns culture identifiers to items.

Copy Task
Describes the Copy task and its parameters.

CreateItem Task
Describes the CreateItem task and its parameters.

CreateProperty Task
Describes the CreateProperty task and its parameters.

Csc Task
Describes the Csc task and its parameters.

Delete Task
Describes the Delete task and its parameters.

Exec Task
Describes the Exec task and its parameters.

FindUnderPath Task
Determines which items in the specified item collection exist in the specified folder and all of its subfolders.

GenerateApplicationManifest Task
Describes the GenerateApplicationManifest task and its parameters.

GenerateBootstrapper Task
Provides an automated way to detect, download, and install an application and its prerequisites.

GenerateDeploymentManifest Task
Describes the GenerateDeployManifest task and its parameters.

GenerateResource Task
Converts .txt and .resx files to common language runtime binary .resources files.

GetAssemblyIdentity Task
Retrieves the assembly identities from the specified files and outputs the identity information.

GetFrameworkPath Task
Retrieves the path to the .NET Framework assemblies.

GetFrameworkSdkPath Task
Retrieves the path to the .NET Framework SDK.

LC Task
Describes the LC task and its parameters.

MakeDir Task
Describes the MakeDir task and its parameters.

MSBuild Task
Describes the MSBuild task and its parameters.

ReadLinesFromFile Task
Reads a list of items from a text file.

RegisterAssembly Task
Describes the RegisterAssembly task and its parameters.

RemoveDir Task
Describes the RemoveDir task and its parameters.

ResGen Task
Describes the ResGen task and its parameters.

ResolveAssemblyReference Task
Describes the ResolveAssemblyReference task and its parameters.

ResolveComReference Task
Describes the ResolveCOMReference task and its parameters.

ResolveKeySource Task
Determines the strong name key source

ResolveNativeReference Task
Resolves native references.

SGen Task
Creates an XML serialization assembly for types in the specified assembly.

SignFile Task
Signs the specified file using the specified certificate.

Touch Task
Describes the Touch task and its parameters.

UnregisterAssembly Task
Describes the UnregisterAssembly task and its parameters.

Vbc Task
Describes the Vbc task and its parameters.

VCBuild Task
Describes the VCBuild task and its parameters.

WriteLinesToFile Task
Writes the specified items to the specified text file.


MSBuild conditions
===================
'A'=='B'      true if A equals B
'A'!='B'      not true if A equals B
<, >, <=, >=    Compares numerical values. Note: need to use < for < and > for >
Exists('sss')   Eg. Condition="!Exists('$(buildir)')"
!             negation
And           and operator
Or            or operator
()            for grouping of expressions



MSBuild reserved properties
============================
Reference: http://msdn.microsoft.com/en-us/library/ms164309(VS.80).aspx

MSBuildProjectDirectory
 The absolute path of the directory where the project file is located, for example, C:\MyCompany\MyProduct.

MSBuildProjectFile
 The complete file name of the project file, including the file name extension, for example, MyApp.proj.

MSBuildProjectExtension
 The file name extension of the project file, including the period, for example, .proj.

MSBuildProjectFullPath
 The absolute path and complete file name of the project file, for example, C:\MyCompany\MyProduct\MyApp.proj.

MSBuildProjectName
 The file name of the project file without the file name extension, for example, MyApp.

MSBuildBinPath
 The absolute path of the directory where the MSBuild binaries that are currently being used are located, for example, C:\Windows\Microsoft.Net\Framework\v2.0. This property is useful if you need to refer to files in the MSBuild directory.

MSBuildProjectDefaultTargets
 The complete list of targets specified in the DefaultTargets attribute of the Project element. For example, the following Project element would have an MSBuildDefaultTargets property value of A;B;C.


MSBuildExtensionsPath
 The MSBuild folder under the Program Files directory. This location is a useful place to put custom target files. For example, your targets files could be installed at \Program Files\MSBuild\MyFiles\Northwind.targets and then imported in project files with the following XML.



MSBuild command line
=====================

For more information on how to use command line, type:
MSBuild.exe /help

Example on a typical usage:
MSBuild.exe /nologo /targets:targetName /property:name=val /verbosity:quiet|minimal|normal|detailed|diagnostic projectFile.proj


MSBuild Well-known item metadata
=================================
Every item has a set of default metadata specified during their creation. The list given below is taken from:
http://msdn.microsoft.com/en-us/library/ms164313(VS.80).aspx


%(FullPath)
 Contains the full path of the item. For example: C:\MyProject\Source\Program.cs

%(RootDir)
 Contains the root directory of the item. For example: C:\

%(Filename)
 Contains the file name of the item, without the extension. For example: Program

%(Extension)
 Contains the file name extension of the item. For example: .cs

%(RelativeDir)
 Contains the directory path relative to the current working directory. For example: Source\

%(Directory)
 Contains the directory of the item, without the root directory. For example: MyProject\Source\

%(RecursiveDir)
 If the Include attribute contains the wildcard **, this metadata specifies the directory that replaced the wildcard to find the item. This example has no RecursiveDir metadata, but if the the following example was used to include this item, the item would contain a RecursiveDir value of MyProject\Source\.




%(Identity)
 The item specified in the Include attribute.. For example: Source\Program.cs

%(ModifiedTime)
 Contains the timestamp from the last time the item was modified. For example: 2004-07-01 00:21:31.5073316

%(CreatedTime)
 Contains the timestamp from when the item was created. For example: 2004-06-25 09:26:45.8237425

%(AccessedTime)
 Contains the timestamp from the last time the time was accessed. 2004-08-14 16:52:36.3168743