Wednesday 17 July 2013

Dual-mode dual-boot Ubuntu with Windows 7 without burning a CD

The term "Dual-mode dual-boot" refers to a single copy of OS which can run either inside Virtual Machine or on its own standalone. You can launch it by either booting directly into it upon machine restart (usually referred to as "dual-boot") or running it inside a virtual machine. This way of installation is very useful for some people like me because Windows is good for playing large-scale computer games while many programming tasks/packages can only be run in Linux environment and all programs running inside virtual machine has performance penalty.

Dual-booting Ubuntu with Windows7 is no longer a big problem for many people. Just a bit of googling will give you tons of solutions. However, to install dual-mode dual-boot Ubuntu on the same harddisk of the currently running Windows7 is kind of headache especially for 64-bit windows because of the new restriction on raw disk access from Vista onwards ( support.microsoft.com/kb/942448 ) as well as Windows 7 x64 forbids loading kernel drivers without Microsoft digital signature (this can be solved by enabling testsigning in boot config using bcdedit).

I have eventually managed to install Ubuntu 12.04 64-bit on my Windows 7 Ultimate 64-bit using VirtualBox 4.2.4 using a Ubuntu ISO image without burning a physical CD. The copy of Ubuntu is installed inside Windows7's VirtualBox but can run not only inside Windows7's virtualBox but also on its own (restart computer and boot directly into the same copy of Ubuntu).

Warning: DO NOT try this if you are NOT an expert in OS boot principle and harddisk partition, attempting things without understanding what exactly it does may result in total loss of your harddisk data and at the same time rendering your current system unbootable. 

Here're the required stuffs you need to prepare:
1. VirtualBox 4.2.4
2. EasyBCD 2.2
3. EaseUS Partition Master (in case needed)
4. Ubuntu CD (12.04 I used)
5. basic knowledge in Command Prompt and Windows disk management
6. in BIOS, enable Intel virtualization technology, otherwise, 64-bit OS cannot be emulated in virtual machine, only 32-bit OS can be emulated

To get started:
1. Make sure you are in administrator mode with UAC disabled, so every operation has system administrator privilege;

2. run the following command in command prompt to enable test signing mode so that some kernel drivers can be loaded (I'm not sure if it's neccesary)
bcdedit.exe -set TESTSIGNING ON

3. in "Windows disk management" (by right-click Computer -> Manage -> Storage -> Disk Management), make sure you have 2 empty partitions prepared for Ubuntu, 1 for system (suggest >=8GB), 1 for swap (total memory size). Your current harddisk type must be a basic disk (not a 'dynamic disk' or a 'GPT disk').
For example, in the figure above, Drive X is for Ubuntu OS and Y is for swap area (optional but strongly recommended unless you don't need virtual memory in Ubuntu).
Take note that the Ubuntu OS better to be installed on a primary partition, swap can be either on primary or extended partition.

4. The two Ubuntu partitions must be unformatted by Windows7, if formatted, delete and recreate; moreover, you MUST assign letters to the two partitions but don't use them in Windows7. In addition, all other partitions must also have be assigned a letter, including the system reserved partition M in my case.

5. Create the disk image. In command prompt, goto virtual box folder,
C:\Program Files\Oracle\VirtualBox in my case, run this:
VBoxManage internalcommands createrawvmdk -filename C:\rawdisk0.vmdk -rawdisk "\\.\PhysicalDrive0" -partitions 0,1,2,3,4,5,6
The image file 'disk0.vmdk' can be put anywhere you want with folder names not containing any space (probably a bug for that version of VirtualBox). The actual number of partitions "0,1,2,3,4,5,6" depends on your harddisk. There must be no errors in running this command, if there's an error, delete the possibly wrongly created disk0.vmdk and disk0-pt.vmdk file and change parameters and retry.

6. Create a new virtual machine in VirtualBox, use this C:\rawdisk0.vmdk as the disk and load Ubuntu ISO image into CD-DVD drive as shown below.

7. Launch the newly created virtual machine, press F12 to boot into Ubuntu CD, perform normal install of Ubuntu, make sure you install into the right partition and set correct swap area. For the boot loader I install into the Ubuntu partition. Try not to install the boot loader into MBR (master boot record) because Windows might not allow you to write MBR directly; also, you may want to remove Ubuntu and free up the 2 partitions at some later time.

Regardless of whether the boot-loader has been installed successfully or not, use EasyBCD to add a new entry and setting the drive to be the Ubuntu partition (i.e. X in my case) as shown below (it will be changed back to C by EasyBCD for some unknown reason):

8. The installation eventually complete successfully and the resultant copy of Ubuntu 64-bit can be run both inside Windows7's virtualBox and upon machine reboot.
Unfortunately, you cannot run the same copy of Windows7 in Linux's virtualBox because the VirtualBox's environment has incurred too many hardware changes that will require another windows activation.


Warning: DO NEVER reboot into the same copy of Windows7 from the Windows7's virtualBox although you can do so in the boot option shown above. Doing so may result in file system corruption or rendering all existing operating systems unbootable.
To prevent doing so, I strongly encourage you to set default system to Ubuntu in the boot option.

The good thing is that the Ubuntu in virtualBox is able to access windows partitions in read-only mode or full-access mode if they are unmounted (in Windows XP or earlier, you can have full access to any partition even when they are mounted; therefore, making changes to that partition from both inside and outside virtual machine causes file system corruption), to unmount a volume E:, run in command prompt:
mountvol
mountvol E:\ /D
after that, mount it in Ubuntu, make some changes to the partition, un-mount it in Ubuntu and re-mount in Windows 7 as follows:
mountvol E:\ VolumeName
Therefore, you need to note down the VolumeName for E:\ when running mountvol for the 1st time.

Here are some of my food-for-thoughts:
1. Although the new restriction in Vista/Win7 forbids full access to \\.\PhysicalDrive0, it does allow full access to individual mounted but unlocked raw partitions, this installation procedure does not violate this rule as long as you install grub loader into the partition boot record rather than MBR

2. Win7 has some harddisk I/O flush delay, updating MBR or partition boot sector takes a long time (say 20 minutes) unless you reboot Win7. So after installation, you might need to reboot your system in order to enter Ubuntu from VirtualBox.

3. In Win7, assigning a drive letter to a partition causes it to be mounted. Every time you make changes to your harddisk, i.e. mount/unmount some partitions, you might need to delete the entire VM in virtualBox manager together with the .vmdk files, and recreate them. Otherwise, it might fail.

I am writing this blog to notify all Linux/Ubuntu OS developers that it's possible for the same copy of OS to run in multiple different environments, e.g. inside VirtualBox or standalone (upon machine reboot). Therefore, OS developer should be aware of certain features and system attributes which might change because of the change in the execution environment.

VisualStudio: custom display datatype in debugger

Edit \Program Files\Microsoft Visual Studio 10.0\Common7\Packages\Debugger\autoexp.dat
Change the following entries for STL:

std::vector<*>{
   children
   (
      #array
      (
         expr :  ($e._Myfirst)[$i],
         size :  $e._Mylast-$e._Myfirst
      )
   )
   preview
   (
      #(
         "[", $e._Mylast - $e._Myfirst , "](",
         #array
         (
            expr : ($e._Myfirst)[$i],
            size : $e._Mylast-$e._Myfirst
         ),
         ")"
      )
   )
}

ATL::CComBSTR{
   preview  ([$e.m_str,su])
   stringview  ([$e.m_str,sub])
}

std::map<*>|std::multimap<*>|std::set<*>|std::multiset<*>{
    children
    (
        #tree
        (
            head : $c._Myhead->_Parent,
            skip : $c._Myhead,
            size : $c._Mysize,
            left : _Left,
            right : _Right
        ) : $e._Myval
    )

    preview
    (
        #(
            "[",
            $e._Mysize,
            "](",

            #tree
            (
                head : $c._Myhead->_Parent,
                skip : $c._Myhead,
                size : $c._Mysize,
                left : _Left,
                right : _Right
            ) : $e._Myval,

            ")"
        )
    )          
}

std::basic_string<char,*>{
preview        ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,s]) #else ( [$e._Bx._Ptr,s]))
stringview    ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sb]) #else ( [$e._Bx._Ptr,sb]))
children
(
#if(($e._Myres) < ($e._BUF_SIZE))
(
#([actual members]: [$e,!] , #array( expr: $e._Bx._Buf[$i], size: $e._Mysize))
)
#else
(
#([actual members]: [$e,!],  #array( expr: $e._Bx._Ptr[$i], size: $e._Mysize))
)
)
}

std::basic_string<unsigned short,*>|std::basic_string<wchar_t,*>{
preview
(
#if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,su] )
#else ( [$e._Bx._Ptr,su] )
)
stringview
(
#if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sub] )
#else ( [$e._Bx._Ptr,sub] )
)

}

VisualStudio: auto-skip STL code when stepping into during debugging

Edit registry, Run regedit.exe

HKLM\SOFTWARE\Microsoft\VisualStudio\10.0\NativeDE\StepOver
(add \Wow6432Node after SOFTWARE if you're on a 64bit machine)

Add a new String Value (REG_SZ). Any name, value=
std\:\:.*=NoStepInto