HardenedBSD/share/doc/handbook/esdi.sgml

<!-- $Id: esdi.sgml,v 1.3 1995/11/25 20:00:45 jkh Exp $ -->
<!-- The FreeBSD Documentation Project -->

<!--
      <title>An introduction to ESDI hard disks and their use with FreeBSD</title>
      <author>(c) 1995, Wilko Bulte, <tt/wilko@yedi.iaf.nl/
      <date>Tue Sep 12 20:48:44 MET DST 1995</date>

    Copyright 1995, Wilko C. Bulte, Arnhem, The Netherlands 

      <abstract>
	This document describes the use of ESDI disks in combination
	with the FreeBSD operating system. Contrary to popular
	belief, this is possible and people are using ESDI based
	systems succesfully! This document tries to explain you 
	how to do this.

	If you find something missing, plain wrong or have useful 
	comments on how to improve
	the document please send mail to <tt/wilko@yedi.iaf.nl/
      </abstract>
-->

    <sect1><heading>Using ESDI hard disks<label id="esdi"></heading>

      <p><em>Copyright &copy; 1995, &a.wilko;.<newline>24 September 1995.</em>

	ESDI is an acronym that means Enhanced Small Device Interface.
	It is loosely based on the good old ST506/412 interface originally
	devised by Seagate Technology, the makers of the first affordable
	5.25" winchester disk.

	The acronym says Enhanced, and rightly so. In the first place
	the speed of the interface is higher, 10 or 15 Mbits/second 
	instead of the 5 Mbits/second of ST412 interfaced drives.
	Secondly some higher level commands are added, making the ESDI
	interface somewhat 'smarter' to the operating system driver
	writers. It is by no means as smart as SCSI by the way. ESDI
	is standardised by ANSI.

	Capacities of the drives are boosted by putting more sectors
	on each track. Typical is 35 sectors per track, high capacity
	drives I've seen were up to 54 sectors/track.

	Although ESDI has been largely obsoleted by IDE and SCSI interfaces,
	the availability of free or cheap surplus drives makes them 
	ideal for low (or now) budget systems.

    <sect2><heading>Concepts of ESDI</heading>
      <p>
      <sect3><heading>Physical connections</heading>
        <p>
	The ESDI interface uses two cables connected to each drive. 
	One cable is a 34 pin flatcable edge connector that carries
	the command and status signals from the controller to the
	drive and viceversa. The command cable is daisy chained
	between all the drives. So, it forms a bus onto which all
	drives are connected.

	The second cable is a a 20 pin flatcable edge connector that
	carries the data to and from the drive. This cable is radially
	connected, so each drive has it's own direct connection to the
	controller.

	To the best of my knowledge PC ESDI controllers are limited
	to using a maximum of 2 drives per controller. This is
	compatibility feature(?) left over from the WD1003 standard
	that reserves only a single bit for device addressing.

      <sect3><heading>Device addressing</heading>
        <p>
	On each command cable a maximum of 7 devices and 1 controller
	can be present. To enable the controller to uniquely 
	identify which drive it addresses, each ESDI device is equipped
	with jumpers or switches to select the devices address.

	On PC type controllers the first drive is set to address 0, 
	the second disk to address 1. <it>Always make sure</it> you
	set each disk to an unique address! So, on a PC with it's
	two drives/controller maximum the first drive is drive 0, the
	second is drive 1.

      <sect3><heading>Termination</heading>
	<p>
	The daisy chained command cable (the 34 pin cable remember?)
	needs to be terminated at the last drive on the chain.
	For this purpose ESDI drives come with a termination resistor
	network that can be removed or disabled by a jumper when it
	is not used.

	So, one and <it>only</it> one drive, the one at 
	the fartest end of the command
	cable has it's terminator installed/enabled. The controller
	automatically terminates the other end of the cable. 
	Please note that this implies that the controller must be 
	at one end of the cable and <it>not</it> in the middle.

    <sect2><heading>Using ESDI disks with FreeBSD</heading>
      <p>
        Why is ESDI such a pain to get working in the first place?
      
	People who tried ESDI disks with FreeBSD are known to have
	developed a profound sense of frustration. A combination of
	factors works against you to produce effects that are
	hard to understand when you have never seen them before.

	This has also led to the popular legend ESDI and FreeBSD
	is a plain NO-GO.
	The following sections try to list all the pitfalls and 
	solutions.

      <sect3><heading>ESDI speed variants</heading>
	<p>
	As briefly mentioned before, ESDI comes in two speed flavours.
	The older drives and controllers use a 10 Mbits/second
	data transfer rate. Newer stuff uses 15 Mbits/second.
	
	It is not hard to imagine that 15 Mbits/second drive cause
	problems on controllers laid out for 10 Mbits/second.
	As always, consult your controller <it>and</it> drive 
	documentation to see if things match.

      <sect3><heading>Stay on track</heading>
	<p>
	Mainstream ESDI drives use 34 to 36 sectors per track. 
	Most (older) controllers cannot handle more than this 
	number of sectors.
	Newer, higher capacity, drives use higher numbers of sectors
	per track. For instance, I own a 670 Mb drive that has
	54 sectors per track.

	In my case, the controller could not handle this number
	of sectors. It proved to work well except that it only
	used 35 sectors on each track. This meant losing a
	lot of diskspace.

	Once again, check the documentation of your hardware for 
	more info. Going out-of-spec like in the example might
	or might not work. Give it a try or get another more
	capable controller.

      <sect3><heading>Hard or soft sectoring</heading>
	<p>
	Most ESDI drives allow hard or soft sectoring to be 
	selected using a jumper. Hard sectoring means that the
	drive will produce a sector pulse on the start of each
	new sector. The controller uses this pulse to tell when
	it should start to write or read.

	Hard sectoring allows a selection of sector size (normally
	256, 512 or 1024 bytes per formatted sector). FreeBSD uses 
	512 byte sectors. The number of sectors per track also varies 
	while still using the same number of bytes per formatted sector. 
	The number of <em>unformatted</em> bytes per sector varies,
	dependent on your controller it needs more or less overhead 
	bytes to work correctly. Pushing more sectors on a track 
	of course gives you more usable space, but might give 
	problems if your controller needs more bytes than the 
	drive offers.

	In case of soft sectoring, the controller itself determines
	where to start/stop reading or writing. For ESDI 
	hard sectoring is the default (at least on everything
	I came across). I never felt the urge to try soft sectoring.

	In general, experiment with sector settings before you install
	FreeBSD because you need to re-run the low-level format
	after each change.

      <sect3><heading>Low level formatting</heading>
	<p>
	ESDI drives need to be low level formatted before they
	are usable. A reformat is needed whenever you figgle
	with the number of sectors/track jumpers or the
	physical orientation of the drive (horizontal, vertical). 
        So, first think, then format. 
	The format time must not be underestimated, for big
	disks it can take hours. 

	After a low level format, a surface scan is done to
	find and flag bad sectors. Most disks have a
	manufacturer bad block list listed on a piece of paper
	or adhesive sticker. In addition, on most disks the
	list is also written onto the disk.
	Please use the manufacturer's list. It is much easier
	to remap a defect now than after FreeBSD is installed.

	Stay away from low-level formatters that mark all
	sectors of a track as bad as soon as they find one
	bad sector. Not only does this waste space, it also
	and more importantly causes you grief with bad144
	(see the section on bad144).
	
      <sect3><heading>Translations</heading>
	<p>
	Translations, although not exclusively a ESDI-only problem, 
	might give you real trouble.
	Translations come in multiple flavours. Most of them 
	have in common that they attempt to work around the
	limitations posed upon disk geometries by the original
	IBM PC/AT design (thanks IBM!).

	First of all there is the (in)famous 1024 cylinder limit.
	For a system to be able to boot, the stuff (whatever 
	operating system) must be in the first 1024 cylinders
	of a disk. Only 10 bits are available to encode the
	cylinder number. For the number of sectors the limit
	is 64 (0-63).
	When you combine the 1024 cylinder limit with the 16 head
	limit (also a design feature) you max out at fairly limited 
	disk sizes. 

	To work around this problem, the manufacturers of ESDI
	PC controllers added a BIOS prom extension on their boards.
	This BIOS extension handles disk I/O for booting (and for
	some operating systems <it>all</it> disk I/O) by using 
	translation. For instance, a big drive might be presented
	to the system as having 32 heads and 64 sectors/track. 
	The result is that the number of cylinders is reduced to
	something below 1024 and is therefore usable by the system
	without problems.
	It is noteworthy to know that FreeBSD after it's kernel has
	started no longer uses the BIOS. More on this later.

	A second reason for translations is the fact that most
	older system BIOSes could only handle drives with 17 sectors
	per track (the old ST412 standard). Newer system BIOSes 
	usually have a user-defined drive type (in most cases this is
	drive type 47).

	<em>Whatever you do to translations after reading this document,
	keep in mind that if you have multiple operating systems on the
	same disk, all must use the same translation</em>

	While on the subject of translations, I've seen one controller
	type (but there are probably more like this) offer the option
	to logically split a drive in multiple partitions as a BIOS
	option. I had select 1 drive == 1 partition because this
	controller wrote this info onto the disk. On powerup it
	read the info and presented itself to the system based on
	the info from the disk.

      <sect3><heading>Spare sectoring</heading>
        <p>
	Most ESDI controllers offer the possibility to remap bad sectors.
	During/after the low-level format of the disk bad sectors are
	marked as such, and a replacement sector is put in place
	(logically of course) of the bad one. 

	In most cases the remapping is done by using N-1 sectors on
	each track for actual datastorage, and sector N itself is 
	the spare sector. N is the total number of sectors physically
	available on the track.
	The idea behind this is that the operating system sees
	a 'perfect' disk without bad sectors. In the case of
	FreeBSD this concept is not usable.

	The problem is that the translation from <it>bad</it> to <it>good</it>
	is performed by the BIOS of the ESDI controller. FreeBSD,
	being a true 32 bit operating system, does not use the BIOS
	after it has been booted. Instead, it has device drivers that
	talk directly to the hardware.

	<em>So: don't use spare sectoring, bad block remapping or
	whatever it may be called by the controller manufacturer when you
	want to use the disk for FreeBSD.</em>
	
      <sect3><heading>Bad block handling</heading>
	<p>
	The preceding section leaves us with a problem. The controller's
	bad block handling is not usable and still FreeBSD's filesystems
	assume perfect media without any flaws.
	To solve this problem, FreeBSD use the <it>bad144</it> tool.
	Bad144 (named after a Digital Equipment standard for bad block
	handling) scans a FreeBSD slice for bad blocks. Having found
	these bad blocks, it writes a table with the offending block
	numbers to the end of the FreeBSD slice. 

	When the disk is in operation, the disk accesses are checked 
	against the table read from the disk. Whenever a blocknumber
	is requested that is in the bad144 list, a replacement block
	(also from the end of the FreeBSD slice) is used.
	In this way, the bad144 replacement scheme presents 'perfect'
	media to the FreeBSD filesystems.

	There are a number of potential pitfalls associated with 
	the use of bad144.
	First of all, the slice cannot have more than 126 bad sectors.
	If your drive has a high number of bad sectors, you might need
	to divide it into multiple FreeBSD slices each containing less
	than 126 bad sectors. Stay away from low-level format programs
	that mark <em>every</em> sector of a track as bad when 
	they find a flaw on the track. As you can imagine, the 
	126 limit is quickly reached when the low-level format is done
	this way.

	Second, if the slice contains the root filesystem, the slice
	should be within the 1024 cylinder BIOS limit. During the
	boot process the bad144 list is read using the BIOS and this
	only succeeds when the list is within the 1024 cylinder limit.
	<em>Note</em> that the restriction is not that only the root
	<em>filesystem</em> must be within the 1024 cylinder limit, but
	rather the entire <em>slice</em> that contains the root filesystem.


      <sect3><heading>Kernel configuration</heading>
        <p>
	ESDI disks are handled by the same <it>wd</it>driver as
	IDE and ST412 MFM disks. The <it>wd</it> driver should work
	for all WD1003 compatible interfaces. 
	
	Most hardware is jumperable for one of two different I/O
	address ranges and IRQ lines. This allows you to have 
	two wd type controllers in one system. 

	When your hardware allows non-standard strappings, you
	can use these with FreeBSD as long as you enter the 
	correct info into the kernel config file.
        An example from the kernel config file (they live in
        <tt>/sys/i386/conf</tt> BTW).

<tscreen><verb>
# First WD compatible controller
controller      wdc0    at isa? port "IO_WD1" bio irq 14 vector wdintr
disk            wd0     at wdc0 drive 0
disk            wd1     at wdc0 drive 1

# Second WD compatible controller
controller      wdc1    at isa? port "IO_WD2" bio irq 15 vector wdintr
disk            wd2     at wdc1 drive 0
disk            wd3     at wdc1 drive 1
</verb></tscreen>

<!--
      <sect3><heading>Tuning your ESDI kernel setup</heading>
        <p>
-->

    <sect2><heading>Particulars on ESDI hardware</heading>
      <p>
	<sect3><heading>Adaptec 2320 controllers</heading>
	<p>
	I succesfully installed FreeBSD onto a ESDI disk controlled by a
	ACB-2320. No other operating system was present on the disk.

	To do so I low level formatted the disk using NEFMT.EXE 
	(<it>ftp</it>able from <it>www.adaptec.com</it>) and answered NO
	to the question whether the disk should be formatted with a
	spare sector on each track. The BIOS on the ACD-2320 was
	disabled. I used the 'free configurable' option in the system
	BIOS to allow the BIOS to boot it.

	Before using NEFMT.EXE I tried to format the disk using the
	ACB-2320 BIOS builtin formatter. This proved to be a showstopper,
	because it didn't give me an option to disable spare sectoring.
	With spare sectoring enabled the FreeBSD installation
	process broke down on the bad144 run.

	Please check carefully which ACB-232xy variant you have. The
	x is either 0 or 2, indicating a controller without or with
	a floppy controller on board. 

	The y is more interesting.  It can either be a blank, 
	a "A-8" or a "D".  A blank indicates a plain 10 Mbits/second
	controller. An "A-8" indicates a 15 Mbits/second controller
	capable of handling 52 sectors/track.
	A "D" means a 15 Mbits/second controller that can also
	handle drives with > 36 sectors/track (also 52 ?).

	All variations should be capable of using 1:1 interleaving. Use 1:1,
	FreeBSD is fast enough to handle it.

    	<sect3><heading>Western Digital WD1007 controllers</heading>
	<p>
	I succesfully installed FreeBSD onto a ESDI disk controlled by a
        WD1007 controller. To be precise, it was a WD1007-WA2. Other
	variations of the WD1007 do exist.

	To get it to work, I had to disable the sector translation and
	the WD1007's onboard BIOS. This implied I could not use 
	the low-level formatter built into this BIOS. Instead, I grabbed
	WDFMT.EXE from www.wdc.com Running this formatted my drive
	just fine. 
	
	<sect3><heading>Ultrastor U14F controllers</heading>
	<p>
	According to multiple reports from the net, Ultrastor ESDI 
	boards work OK with FreeBSD. I lack any further info on
	particular settings.
	
<!--

    <sect2><heading>Tracking down problems</heading>
      <p>
-->

    <sect2><heading>Further reading<label id="esdi:further-reading"></>
      <p>
        If you intend to do some serious ESDI hacking, you might want to
        have the official standard at hand:

        The latest ANSI X3T10 committee document is:
        <itemize>
<item>Enhanced Small Device Interface (ESDI) &lsqb;X3.170-1990/X3.170a-1991&rsqb;   
                                   &lsqb;X3T10/792D Rev 11&rsqb;
        </itemize>
        On Usenet the newsgroup <htmlurl url="news:comp.periphs"
        name="comp.periphs"> is a noteworthy place to look 
	for more info.

	The World Wide Web (WWW) also proves to be a very handy info source:
	For info on Adaptec ESDI controllers see <htmlurl
        url="http://www.adaptec.com/">.
	For info on Western Digital controllers see <htmlurl
        url="http://www.wdc.com/">.

    <sect2>Thanks to...
	<p>
	Andrew Gordon for sending me an Adaptec 2320 controller and ESDI disk 
	for testing.