Showing posts with label How-To Diskless AoE. Show all posts
Showing posts with label How-To Diskless AoE. Show all posts

Dec 19, 2014

Windows Diskless with AoE Server Linux Distro

This is the new project for the community. A based Linux Distro ready-to-run for Diskless Server !
Windows Diskless with AoE Server Linux Distro

Main Features:
  • Simple DHCP Server configuration powered with a opensource project called "Opendhcpd Server"
  • MagicVolume with powered cool feature with Deduplicated FileSystem ! Now you can deploy more than 100 Windows diskless clients with ( for example ) a single SSD 128Gb Hard Disk, this feature is powered by opensource project OpenDedup.org !
  • Web Interface for Administration powered by opensource project Webmin;
  • Built-in ‘vblade‘ for easy export virtual disks (.vhd, .vmdk files)
  • Easy Windows Share access to Upload your Master Image directly to the MagicVolume;
Preview release for Download:

http://susestudio.com/a/UZQFsW/windows-diskless-with-aoe

Links:

- How to install the Distro
- The Server Specs
- Configure a Bond Interface
- The required Services
- The MagicVolume How-To
- The TFTP Server

StarWind AoE Initiator

ATA-over-Ethernet Initiator for Microsoft Windows

ATA-over-Ethernet (AoE) is a network protocol designed for simple high-performance access to SATA storage devices over Ethernet. AoE does not rely on network layers above Ethernet, such as IP and TCP. It is more comparable to Fibre Channel over Ethernet than iSCSI.


AoE Initiator fully implements ATA-over-Ethernet client side protocol enabling you to connect to remote storage appliances such as Coraid®. AoE support enables connectivity to remote storage devices over a standard Ethernet network, regardless of where the device resides. With AoE Initiator and ATA-over-Ethernet, you can gain full access to a device anywhere on an Ethernet network, as though it were installed on a local machine.

Download StarWind AoE Initiator here

WinAoE Driver


WinAoE Driver is from http://www.winaoe.org

You should download this version because is already self sign files

Official Web Site: http://www.winaoe.org

Screenshots:

Game Disk Creation

To complete your Diskless Solution for you CyberCafé, you need a Game Disk !

Game disk is in fact a virtual disk. It’s a physical hard disk or just a partition on server. Game disk is almost used for game disk – on server, install games into this disk, on client side, you will get this disk and run games directly just like a local disk.
Of course, you can also use this function for other intentions.

The size of Game Disk will be very high.

We can create a Game Disk with VirtualBox to be a Master Image, after prepare all games, you can make multiples copies of this image to each Client (computer).

Note: On your Diskless Setup, need to exist one Game Disk per Client ( Computer );

Procedure
  • Creating a Game Disk
  • Export Game Disk with vblade
  • Install AoE Initiator
  • Attach Disk with AoE Initiator
  • Format Disk with Windows
  • Dettach Disk
  • Attach Disk on Master Client Machine ( image )
1 – Creating a Game Disk

With VirtualBox, open a settings from any Virtual Machine , and on Storage Section create a Disk.

Task 1 – Create disk

Task 2- Select format

Task 3 – Fixed Size

Task 4 – Set name and location

Task 5 – Finish

Task 6 – Wait the creation ( 10 minutes )

2 – Export Game Disk with vblade

Use vblade to export the AoE Targe.
vblade -b 400 1 1 "\Device\{value}" "d:\Master-Game-Disk.vhd"

3 – Install AoE Initiator

Download and install “Aoe Initiator” from StarWind Software.

The installation is simple, just Next, Next, Finish…

4 – Attach Disk with AoE Initiator

Attach the disk, set name and mark options.

5 – Format Disk with Windows

Open the “Disk Management Utility” from Windows and create a partition and format the Drive.

6 – Dettach Disk

Dettach the Disk;

7 - Attach Disk on Master Client Machine ( image )

Now attach the Game Disk on Client, install AoE initiator ! Your Game Disk is ready.

Install all games, after disconnect AoE Initiator and on server, multiply copies of game disk to other clients.

Note: After connect on disk, put the AoE Initiator icon on Startup of Windows to automatically map the new Game Disk on boot.

Note 2: Never connect exported disk with two or more AoE Initiators with differents machines at same disk simultaneously.
     

Control your vBlade’s Sessions with nice Interface

You can control your vblade sessions with a simple and nice interface to manage:

You can run the vblade session without a console with free “System Scheduler

Oficial website: http://www.splinterware.com/products/wincron.htm

Alternative Download Link: Ssfree.exe


and kill the process with “Process Explorer” from SysInternals Microsoft

Cache Write-Back

For cache write back feature you can use:

FancyCache is a supplementary software caching scheme that cooperates with system memory to provide data caching for volumes/disks. It improves system performance by transparently storing data into memory such that future requests for that data can be served faster. FancyCache caches data on a logical block basis (offsets within a volume/disk) while windows cache manager caches on a virtual block basis (offsets within a file).


FancyCache can also utilize the OS Invisible Memory which is wasted on 32-bit Windows when 4GB memory or more is installed.

Main Features
  • Supports LRU (Least Recently Used) and LFU (Least Frequently Used) cache algorithms
  • Supports caching strategies: Read/Write Caching, Read-Only Caching and Write-Only Caching
  • Supports Write-Through and Write-Deferred modes
  • Supports OS Invisible Memory and SSD (Solid-state Drive) as Level-II cache
  • Supports caching for volumes or entire disks
  • Supports TRIM command for SSD
  • Supports visual performance monitor
  • Supports cache plug and play
  • Supports basic and dynamic disks
  • Supports NTFS junction point
  • Supports volume/disk with proprietary file system
Website: https://web.archive.org/web/20131117170301/http://www.romexsoftware.com/en-us/fancy-cache/index.html

Download:

FancyCache Volume Edition: Download Sites

FancyCache Disk Edition: Download Sites

Coraid

Coraid is High Performance and Low Cost Networked Storage

Coraid is redefining the fundamental economics of storage with Ethernet SAN solutions that provide enterprises of all sizes with flexible, high-performance, scale-out storage ideally suited to dynamic High Performance Computing, video, virtualization and cloud environments. The key is Coraid’s use of the advanced protocol ATA over Ethernet.

Business Benefits
  • Large amounts of shared storage at low prices.
  • Better performance than traditional storage technology.
  • Especially good for storing video and deploying Virtual Desktop Environments.
  • Over 1300 Satisfied Customers of Coraid including Ford and Disney.
  • New storage access technology means lower price and higher performance with better reliability.
Technical Features and Benefits
  • 5-8x Price/Performance advantage over Fibre Channel and iSCSI storage arrays.
  • CorOS™ parallel processing scale-out SAN operating system.
  • Distributed, modular architecture scales from a few Terabytes to Petabytes.
  • Automatic Load Balancing reduces bottlenecks; ideal for VMware environments.
  • Continuous RAIDShield™ testing fixes disk bad blocks; reduces drive failures.

New Technology Designed for Rapid Access at Low Cost (see above)
  • Layer-2 Ethernet
  • Less than $1,000/TB
  • Incredibly fast (>1,800 MB/s)
  • Plug-and-play configuration
  • Scale-out architecture
Better Than Traditional Fibre Channel or iSCSI Networked Storage (see above)
  • Expensive
  • Controller hurts performance
  • Complex to set-up and maintain
  • Difficult to scale

Simplified SAN Management

Coraid’s CorOS™ software provides the intelligence that enables rapid installation (minutes not hours), simple management, and RAID redundancy. The simplicity of EtherDrive Storage makes it easy to manage, because it’s easy to understand.

EtherDrive is a disk connected to your server via Ethernet. That disk may actually be a RAID volume, but it just looks like a local disk to the server. No need to learn complicated technology like Fibre Channel. Just add SAS, SSD or SATA disks to the EtherDrive Storage Appliance and you have all the space you want available for any server attached to the Ethernet SAN. It’s that simple.

Disruptive Price/Performance

Coraid’s EtherDrive® Ethernet SAN storage solutions deliver flexible, high-performance, scale-out storage designed from the ground up for simplicity and virtualization. EtherDrive® storage enables lower OPEX and 5-8x price performance advantage over legacy Fibre Channel systems.

How-To Diskless AoE – 09 Preparing a Real PC Lan Booting

To boot a real PC you should need to Convert a Windows installation to a .vhd, .vmdk or .img file, but in some cases you can use the VirtualBox Windows installtion to boot real machines ! So maybe you dont need to Convert.

The cases are if the NIC Card is recognized by operating system Windows XP/7/8 by default. Verify your onboard NIC and mark if that exists on default Windows drivers list. The CCBoot(r) Client have a PNP tool that list and collect the NIC Drivers from Windows system and put this nic drivers available to diskless boot.


See the CCBootPnP:


Click in the 3 buttons to apply the Drivers…  Now procedd to try boot your .vhd or .vmdk file from Diskless Server.

If Windows loading freeze,

 you need to Convert a real Installation, try this procedure:
Converting the Real PC with Windows installed ( apply to xp, 7, 8, etc…)

On PC Windows Station:
Task 1 – Download HDClone http://www.miray.de/products/sat.hdclone.html and install.
Task 2 – Download AoE Initiator http://www.starwindsoftware.com/aoe-ataoverethernet-initiator and install.
 
On Diskless Server:

Task 3 – Create a virtual disk with a required Windows size (that can support the actual partition size of your current Windows Station). You can use the graphical VirtualBox Manage to create .VHD 

Files, or use the ‘VBoxManage.exe’ tool.

If you running in a Windows Server with VirtualBox installed, you will have the VBoxManage.exe command:


example creating 25 Gb of disk:

On Windows:


On Linux Distro (version 0.0.15+):


Task 4 – on server export as a AoE target with vblade (Windows Server or Linux Distro method).

On PC Windows Station:

Task 5 - Use AoE Initiator from StartWind to connect to the exported aoe disk by Diskless Server, with  ”Automount the device” option unmarked.

Vblade tool from Windows Diskless Server exporting a disk:

The disk wil be connected:


Task 6 - Use HDClone — Clone hdd physical to virtual attached disk by AoE Initiator.


Task 7 – Validate the Conversion ( cloned ), initialize/activate the disk on ‘Disk Managment’ on ‘Computer Manager’, after Windows recognize the disk, view the disk by Windows Explorer.


Task 8 - Remove the physical HDD from PC Station and try to Boot From Diskless Server;

How-To Diskless AoE – 08 Editing with notepad++ the menu file ‘menu.lst’ of TFTP Root

Now its time to configure the Boot Menu of your Diskless Solution.

We already copy the necessary files to TFTP Root folder ( grldr, the new ipxe.iso and menu.lst) .

Open with Notepad++ the file ‘menu.lst‘.

Erase all the contents, copy and paste this code block above:

title === MENU BOOTS ===
()
title
()
title Windows 7 Diskless
map --mem (pd)/ipxe.iso (0xff)
map --hook
chainloader (0xff)

Now start, DHCP Server and TFTP Server and test the system with VirtualBox with a machine without disks.

Of course Boot from LAN:

Your menu will open:

Press <ENTER> and see the magic !


Select the respective option and go, your Windows Diskless will be booted.

Thats all ! Cya

To prepare a Real PC , go to Step 9

Dec 18, 2014

How-To Diskless AoE – 07 Rebuilding the file ipxe.iso with UltraISO

Now its time to create the POINTER to the AoE Target to automaticaly boot.


* – watch in HD on Youtube.

We are using the ipxe.iso with a little additions to do this.

You will need download the ipxe.isoUltraISO and Notepad++ Software to complete this task.

Task 1 – On UltraISO open the file ipxe.iso


Drag the file ISOLINUX.CFG to outside UltraISO window and edit this file with notepad++, put the last line with ‘INITRD conf.ipxe‘,  save and drag the file again into UltraISO. Overwrite YES.
# These default options can be changed in the geniso script
SAY iPXE ISO boot image
TIMEOUT 30
DEFAULT ipxe.lkrn
LABEL ipxe.lkrn
KERNEL ipxe.krn
INITRD conf.ipxe

Task 2 – With Notepad++ create the file  ’conf.ipxe‘  with content:
#!ipxe
dhcp net0
set keep-san 1
sanboot aoe:e0.0

Important ! Save the file conf.ipxe with notepad++, and drag the file backing into UltraISO.

Your UltraISO looks like this:


Task 3 – Save another ISO

On UltraISO menu, go to ‘Save as’ and save ipxe.iso into your TFTP Root folder.

Now you have a ipxe.iso with aoe target address (ex. e0.0) to boot.

Its time to create a PXE Menu to boot your new ipxe.iso !

Proceed to the Step 8

How-To Diskless AoE – 06 Installing and Configuring vBlade Target System

Now its time to install the Target AoE Server, we will use the vBlade ported to Windows to start.


* – Watch in HD on Youtube 

With this software we will export a aoe target (like e0.0) with the virtual disk file ( .VHD or .VMDK ) from the prepared Windows 7 that we installed on Virtual Box.

Important: To vblade work properly on your Server you should have WinPCAP installed already.

After download the installer

Task 1 – Install:





Configuring

Task 1- Open the icon ‘Run vBlade’ on Program Group


Task 2 - on cmd window export your new .VHD or .VMDK Windows Virtual Machine to AoE Target !

Type:

First find the Device Name of your Network Card with bogus caracters (xxx).
c:\vblade.exe -b 65 0 0 xxx "d:\path_to_you_vhd\windows.vhd"

after copy and paste the \Device\{value}


c:\Program Files (x86)\vBlade\vblade.exe -b 65 0 0 "\Device\{value}" "d:\path_to_you_vhd\windows.vhd"


*- where \Device\{value} of your NIC !

Your target is e0.0 ( shelf 0 slot 0 , you can export many .vhd or .vmdk files to many diskless workstations )

ok, now you have your Target UP !

Tip: Control your vBlade’s Session with a nice Interface

Note: Make sure your Windows Firewall are disabled and your Anti-Virus Protection will not block the connections.

Continue to Step 7

How-To Diskless AoE – 05 Instaling a virtual machine in VirtualBox with WinAoE driver and CCBoot Client

Now we gonna change the focus, to prepare a Virtual Machine with VirtualBox to test our environment while we configuring the the Diskless Server.


* – Watch in HD on Youtube
- So, first Install VirtualBox, and create a VirtualMachine to Install one Windows. We gonna choose now the Windows 7 ok ?

Task 1 – Install Windows 7 VirtualMachine.


You should have the .ISO DVD from your Windows to install with Virtual Box.

Specifications for Virtual Machine:  Standard, 1 or 512 mb memory, Disk 25 gb, etc.

Note: Create the disk with .VHD or .VMDK format with fixed size !

- Install WinAoE Driver

Task 1 - To driver be loaded we should disable the Integrity checks for sign certification of drivers.

( Note: Windows XP not need to use this two commands, the driver will install clearly, proceed… Task 3 )

In Windows open a cmd ( command prompt ) and type
C:\>bcdedit  -set  TESTSIGNING ON
C:\>bcdedit -set loadoptions DDISABLE_INTEGRITY_CHECKS


Task 2 - reboot Windows ( mandatory )

Task 3 - Download the driver here, because this version is self signed : winaoe.zip

Task 4 - Uncompress file into a folder

Task 5 - Go to Start Button -> Run –> type ‘hdwwiz.cpl’

Task 6 - Install driver fwith type Storage Adapter from aoe.inf


Task 7 – reboot Windows

Task 8  - Open ‘Device Manager and see is the AoE Driver is correctly runnig


Open a cmd ( command prompt ) and type command ‘aoe’ , type command ‘aoe scan’, the result must be ‘No AoE targets found’, your AoE Driver is working !


 - Install CCBoot Client

After install Windows, you should download the CCBoot Client ,install it and Run It to complete the installation on Virtual Machine with this settings. ( no mark options ).

Task 1- Click on “Install CCBoot Client” button to finish.  Power off the Virtual Machine.


Task 2 - Shutdown the Virtual Machine.

Proceed on How-To to Step 6

How-To Diskless AoE – 04 Putting the necessary files of grub4dos on TFTP Root folder

After download the grub4dos package from the project website. Open it and extract the files grldr and menu.lst and put on  your  tftp-root folder.


* – watch video in HD.

Task 1 – Extract two files


Task 2 – Your tftp-root folder should be like this, with this two files (+ ipxe.iso will see later):


Proceed on the How-to to the next Step 5

How-To Diskless AoE – 03 Instaling and configuring TFTP Server on Server

Choose your TFTP Server from the suggestions.

Im using a TFTP Server from SolarWinds, its completly freeware and works fine.  To explain configuration im using this package. But as you know, you can done with other.

Installing

Task 1  - <INSTALL>


Task 2  -  <NEXT>


Task 3  - <NEXT>

Task 4  - waiting…


Task 5  - <FINISH>


Task 6  - Click on Start Windows button and find SolarWinds TFTP menu program group. Open the program.


Task 7  - Menu “File” -> “Configure”


Task 8  - Configure like image, and choose your own tftp-root folder.

Proceed on How-To to next Step 4 - Putting the necessary files ( grub4dos )

How-To Diskless AoE – 02 Instaling and configuring DHCP Server on Server

* – watch video in HD, click on Youtube

Im using a DHCP from Uwe.Ruttkamp. The explain configuration im using this package. But as you know, you can done with other. Just put the option bootfilename string as ‘glrdr‘. ( alternative download link: glrdr )

Its very simple and have with a nice Wizard. To start click on ‘dhcpwiz.exe‘


Task  1:  Just click in <Next>;


Task  2: Select your internal network interface;


Task  3: If you want to use the TFTP from this package, mark TFTP option and choose a folder. Note: on the root path folder put the file ‘grldr‘ and ‘menu.lst‘. If NOT just Click in <NEXT>;


Task 4: Write the string ‘grldr’ on bootfile option and your internal domain. Click in <NEXT>;


Task 5: Mark overwrite and click on “Write INI file” and <NEXT>;
 
Task 6: Start the Service. and <FINISH>;

Done

If you want to use other package for TFTP continue to the step 3, if NOT skip to Step 4.

How-To Diskless AoE – 01 Overview of the Solution

This How-To enable a Windows based solution that uses AoE technology to bring an entirely new range of solutions, flexibility and cost reductions to businesses. The feature of AoE is a server based network where software applications and programs are held on the server, and runs on Client PCs (Diskless Node). Therefore, Client PCs do not require a hard disk anymore. Centralizing operating system data by deploying AoE enables storage virtualization at the level of the local hard drive, and allows extremely fast server and desktop deployment. This makes AoE Diskless an ideal network management software which is suitable for all kinds of networked environment industries such as Education Institutions, Training Centers, Offices, Cybercafé, Karaoke, and can also be used in cluster computing.

Until today, administrators or technical support staffs are still having frustrations when it comes to troubleshooting and maintaining a group of networked PCs. The majority of problems faced by administrator or technical support staff in a networking environment are:-
  • Programs/Applications/Games/Windows Updates to all PCs
  • Maintain different PC specification
  • Efficiency and Troubleshooting of PCs
  • Identifying faulty hardware and replacements
  • Hard disk limitation and upgrades
  • Virus attacks and Virus removal
  • Operating System Backup / Restoration
  • Windows / Files Protection
  • Freeze/Unfreeze PCs when doing updates (Recovery system)
Listed below are some quick facts if you use this How-To:

COST SAVING IN:
  • Investment for hard disk and future hard disk upgrade
  • Monthly electricity bill, go Green
  • Recovery software / hardware
  • Backup / cloning software and other update software
  • Antivirus / Anti Trojan software
  • Faulty hard disks replacements
TIME SAVING IN:
  • Programs/Applications/Games/Windows Update to all PCs
  • PC maintenance enabling easy manage on multiple branches remotely
  • Virus attacks and Virus removal
  • Windows / Files Protection
  • Maintaining different specification PCs
  • Operating System Backup / Restoration
  • Freeze/Unfreeze (Recovery system)
SUPPORT:
  • Different Client PC specification with different drivers (Motherboard / Display / Sound / etc)
  • Multi Restore Points
  • Multi Sync between Servers
  • Multiple Images – Multiple Window. (Example: 10PC using English Windows + 10PC Malay Windows + 10PC Chinese Windows)
What is AoE ?

ATA over Ethernet (AoE) is an open standards based protocol that allows direct network access to disk drives by client hosts. Using disk storage arrays that support AoE shared storage networks (SAN) can be built that leverage the power of “Raw” Layer 2 Ethernet.
  • AoE has been native in the Linux kernel since 2005
  • AoE delivers a simple, high performance, low cost alternative to iSCSI and FibreChannel for networked block storage by eliminating the processing overhead of TCP/IP.
  • Layer 2 Protocol which encapsulates ATA (the command set used by most commodity disk) in Ethernet Frames – An Ethernet request which has in it, give me block ‘00’ from disk ‘01’ on shelf ‘1’.
Protocol

AoE is a stateless protocol which consists of request messages sent to the AoE server and reply messages returned to the client host.

Messages have two formats:
  • ATA Message
  • Config/Query Messages
AoE utilizes the standard Ethernet MAC header for IEEE 802.3 Ethernet frames and has a registered Ethernet type of 0x88A2.

Legacy Fibre Channel and iSCSI protocols consist of several complex software layers see the diagram below. These layers force users through mandatory SAN point-to-point connection configuration procedures for each network path for all storage LUNs. Ethernet SAN is a connectionless protocol that connects servers and storage directly across layer 2 Ethernet. It does not require TCP/IP or user configured multi-path IO (MPIO) software. The use of layer 2 Ethernet represents a simpler approach for SAN.

How-To Boot Windows Diskless with AoE instead iSCSI

1.] Preparing Windows to Boot Diskless with VirtualBox/Vmare Workstation/ESX
  • Install Windows 7/8/2003/2008 on a Virtual Machine ( create a disk type .vhd with fixed size )
  • open a cmd as administrator and type two commands:
  • C:>bcdedit  -set  TESTSIGNING ON
    C:>bcdedit -set loadoptions DDISABLE_INTEGRITY_CHECKS
    
  • reboot the /virtual machine
  • Download and Install WinAoE Driver as Storage Controller
  • Download and Install CCBoot Client without options
  • Shutdown Virtual Machine
2.] On Server
  • Install a DHCP Server with option bootfilename grldr
  • Install a TFTP Server and set tftp-root folder
  • Download grub4dos and extract 'grldr' and 'menu.lst' files to the tftp-root folder
  • Edit menu.lst file only with this content:
  • title === MENU BOOTS ===
    ()
    title
    ()
    title Windows 7 Diskless
    map --mem (pd)/ipxe.iso (0xff)
    map --hook
    chainloader (0xff)
    
  • Create a text file 'conf.ipxe' with this content:
  • #!ipxe
    dhcp net0
    set keep-san 1
    sanboot aoe:e0.0
    
  • Download ipxe.iso and with UltraISO, open the ipxe.iso with UltraISO and edit ISOLINUX.CFG with this content:
  • SAY iPXE ISO boot image
    TIMEOUT 30
    DEFAULT ipxe.lkrn
    LABEL ipxe.lkrn
    KERNEL ipxe.krn
    INITRD conf.ipxe
    
  • Update the ISOLINUX.CFG on UltraISO with new content
  • Drag the file 'conf.ipxe' to inside UltraISO and save the file.
  • Put ipxe.iso on the tftp-root folder
  • Start DHCP Server
  • Start TFTP Server
  • Download and install WinPCAP and vBlade Target System ( AoE ) and export the target of .vhd.
  • Open vblade from Icon on Program Group
  • First find the Device Name of your Network Card with bogus caracters (xxx).
  • vblade -b 65 0 0 xxx “d:path_to_you_vhdwindows.vhd”
  • after copy and paste the Device{value}
  • vblade -b 65 0 0 “Device{value}” “d:path_to_you_vhdwindows.vhd”  enter 
  • Ready ! Boot one machine by LAN Boot, or create a VirtualBox/Vmware Virtual Machine without Disks !
  • You can boot a lot of Machines creating a specific ipxe.iso for each one, dont forget to edit 'conf.ipxe' with diferent targets ( e0.0, e0.1, e0.01), create respective ipxe-e0.0.iso for each workstation, add the menu.lst entry too for each ipxe-e0.x.iso, export each .vhd with vblade changing slots ( 0 1 , 0 2 etc..)
via http://windowsdisklessaoe.wordpress.com

Dec 13, 2014

List of Diskless Booting Software



List of Diskless Booting Software for iCafe and LANshop

CCBoot by YoungSoft ( Comercial ) - iSCSI based

iShareDisk ( Free ) - iSCSI based

RichTech Diskless by RichTech ( Comercial ) - iSCSI based

Serva Diskless Installation System ( Comercial ) - iSCSI based

OBM Diskless by OBM ( Comercial ) – iSCSI based

EMS358 by EMS ( Chinese, comercial )

Depth Internet Diskless System ( Chinese, comercial )

KeyDone Diskless ( Comercial )

NxD Diskless ( Comercial ) Free

WinTarget AoE Server ( Free 1 connection / Comercial )

DDS Diskless Solution for Cybercafe | MichaelSoft

Orb Diskless

SanDeploy

Diskless Remote Boot in Linux (DRBL)

Q. Q. Diskless Solutions | links

Dec 12, 2014

Yet Another AoE vs. iSCSI Opinion

That’s right, folks! Yet another asshole blogger here, sharing his AoE (ATA over Ethernet) vs. iSCSI (Internet SCSI) opinion with the world!

As if there wasn’t already enough discussion surrounding AoE vs. iSCSI in mailing lists, forums and blogs, I am going to add more baseless opinion to the existing overwhelming heap of information on the subject. I’m sure this will be lost in the noise but after having implemented AoE with CORAID devices and iSCSI with an IBM (well, LSI) device and iSCSI with software targets in the past I feel I finally have something share.

This isn’t a technical analysis. I’m not dissecting the protocols nor am I suggesting implementation of either protocol for your project. What I am doing is sharing some of my experiences and observations simply because I can. Read on, brave souls.

Background

My experiences with AoE and iSCSI are limited to fairly small implementations by most standards. Multi-terabyte and mostly file serving with a little bit of database thrown in there for good measure. The reasoning behind all the AoE and iSCSI implementations I’ve setup is basically to detach storage from physical servers to achieve:
  1. Independently managed storage that can grow without pain
  2. High availability services front-end (multiple servers connecting to the same storage device(s))
There are plenty of other uses for these technologies (and other technologies that may satisfy these requirement), but that’s where I draw my experiences from. I’ve not deployed iSCSI or AoE for virtual infrastructure which does seem to be a pretty hot topic these days, so if that’s what you’re doing, your mileage will vary.

Performance

Yeah, yeah, yeah, everyone wants the performance numbers. Well, I don’t have them. You can find people comparing AoE and iSCSI performance elsewhere (even if many of the tests are flawed). Any performance numbers I may accidentally provide while typing this up in a mad frenzy are entirely subjective and circumstantial… I may not even end up providing any! Do you own testing, it’s the only way you’ll ever be sure.

The Argument For or Against

I don’t really want to be trying to convince anyone to use a certain technology here. However, I will say it: I lean towards AoE for the types of implementations I mentioned above. Why? One reason: SIMPLICITY. Remember the old KISS adage? Well, kiss me AoE because you’ve got the goods!

iSCSI has the balls to do a lot, for a lot of different situations. iSCSI is routable in layer 3 by nature. AoE is not. iSCSI has a behemoth sized load of options and settings that can be tweaked for any particular implementation needs. iSCSI has big vendor backing in both the target and the initiator markets. Need to export an iSCSI device across a WAN link? Sure, you can do it, never mind that the performance might be less than optimal but the point is it’s not terribly involved or “special” to route iSCSI over a WAN because iSCSI is designed from the get-go to run over the Internet. While AoE over a WAN has been demonstrated with GRE, it’s not inherent to the design of AoE and never will be.

So what does AoE have that iSCSI doesn’t? Simplicity and less overhead. AoE doesn’t have myriad of configuration options to get wrong, it’s really so straight forward that it’s hard to get it wrong. iSCSi is easy to get wrong. Tune your HBA firmware settings or software initiator incorrectly (and the factory defaults can easily be “wrong” for any particular implementation) and watch all hell be unleashed before your eyes. If you’ve ever looked at the firmware options provided to by QLogic in their HBAs and you’re not an iSCSI expert, you’ll know what I’m talking about.

Simplicity Example: Multipath I/O

A great example of AoE’s simplicity vs. iSCSI is when it comes to multipath I/O. Multipath I/O is defined as utilizing multiple paths to the same device/LUN/whatever to gain performance and/or redundancy. This is generally implemented with multiple HBAs or NICs on the initiator side and multiple target interfaces on the target side.

With iSCSI, every path to the same device provides the operating system with a separate device. In Linux, that’ll be /dev/sdd, /dev/sde, /dev/sdf, etc. A software layer (MPIO) is required to manage I/O across all the devices in an organized and sensible fashion.

While I’m a fairly big fan of the latest device-mapper-multipath MPIO layer in modern Linux variants, I find AoE’s multipath I/O method much, much better for the task of providing multiple paths to a storage device because it has incredibly low overhead to setup and manage. AoE’s implementation has the advantage that it doesn’t need to be everything to every storage subsystem, which fortunately or unfortunately device-mapper-multipath has to be.

The AoE Linux driver totally abstracts multiple paths in a way that iSCSI does not by handling all the multipath stuff internally. The host is only provided with a single device in /dev that is managed identically to any other non-multipath device. You don’t even need to configure the driver in any special way, just plug in the interfaces and go! That’s a long shot from what is necessary with MPIO layers and iSCSI.

There’s nothing wrong about device-mapper-multipath and it is quite flexible, but it certainly doesn’t have the simplicity of AoE’s multipath design.

Enterprise Support

Enterprise support is where iSCSI shines in this comparison. Show me a major storage vendor that doesn’t have at least one iSCSI device, even if they are just rebranded. Ok, maybe there are a few vendors out there without an iSCSI solution but for the most part all the big boys are flaunting some kind of iSCSI solution. NetApp, EMC, Dell, IBM, HDS and HP all have iSCSI solutions. On the other hand, AoE only has only a single visible company backing it at the commercial level: CORAID, a spin-off company started by Brantley Coile (yeah, the guy who invented the now-Cisco PIX and AoE). I’m starting to see some Asian manufacturers backing AoE on the hardware level but when it comes to your organization buying rack mount AoE compatible disk trays, CORAID is the only vendor I would suggest at this time.

This isn’t so fantastic for getting AoE into businesses but it’s a start. With AoE in the Linux kernel and Asian vendors packing AoE into chips things will likely pickup for AoE from an enterprise support point of view: It’s cheap, it’s simple and performance is good.

Conclusion

AoE rocks! iSCSI is pretty cool too, but I’ve certainly undergone much worse pain working with much more expensive iSCSI SAN devices vs the CORAID devices. And no performance benefit that I could realize with moderate to heavy file serving and light database workloads. I like AoE over iSCSI but there are plenty of reasons not to like it as well.

ATA-over-Ethernet vs iSCSI

Every so often someone voices interest in ATAoE support for Solaris or tries to engage in an ATAoE versus iSCSI discussion. There isn't much out there in the way of information on the topic so I'll add some to the pot...

If you look just at the names of these two technologies you can easily start to equate them in your mind and start a running mental dialog reguarding which is better. But, most folks make a very common mistake.. ATA-over-Ethernet is exactly that, over ethernet. Whereas iSCSI is Internet SCSI, or as some people prefer to think SCSI over IP. So we've got two differentiators just given the names of these technologies alone: ATA vs SCSI command set, and Ethernet vs IP stack. The interesting thing is the latter discussion.

There is a natural give and take here. The advantage of ATAoE is that you don't have the overhead of translating ATA to SCSI then back to ATA if your using ATA drives, so there is a performance pickup there. Furthermore, because we don't have the girth fo the TCP/IP stack underneight we don't burden the system with all that processing, which adds even more performance. In this sense, ATAoE strips away all the stuff that gets in the way of fast storage over ethernet. But, naturally, there is a catch. You can't route ethernet, thats what TCP/IP is for. That means that with ATAoE your going to be building very small and localized storage networks on a single segment. Think of a boot server which operates without TCP/IP, you've got to have one per subnet so that it see's the requests.

iSCSI on the otherhand might be burdened by the bulk of the TCP/IP stack, however it has the ability to span the internet because of it. You can have an iSCSI target (server) in New York and an iSCSI initiator (client) in London connected across a VPN and its not a problem. Plus, iSCSI is an open and accepted standard. ATAoE on the otherhand is open but it was created and developed by Coraid who also happens to be the only supplier of ATAoE enclosures. That may change, but we'll see how well it catches on.

ATAoE promises to be smaller and faster than the industry standard iSCSI, and it is, but unless you are using a very local application your going to be in trouble. Not to mention the lack of enclosure and driver support for non-Linux systems.

The question then becomes: Should OpenSolaris support ATAoE? Personally, I don't think we should ever be against the idea of anything new, if someone wants to do it, we should all get behind it. But looking at Solaris I doubt the idea would stick. First and foremost Solaris is an OS that adheres to the standards and plays by the rules, even when its painful. Linux doesn't always play by those rules and often it gains from breaking them. Linux is a great experimental platform, no doubt, but I just don't think the ideals of ATAoE mesh well with the goals of Solaris. Furthermore, ATAoE doesn't offer the level of scalability, flexablilty, and managability that we get with iSCSI. The performance hit of TCP/IP is definately a downside, but the advantages it brings to the table far out weight the downsides I think.

Here are some links to help you explore the subject more on your own: