Dr. Jerry Pournelle

Email Me


Why not subscribe now?

Chaos Manor Subscribe Now



Useful Link(s)...

JerryPournelle.com


Hosting by
  Bluehost


Powered by Apache

Computing At Chaos Manor:
The Mailbag

Mailbag for January 2, 2007
Jerry Pournelle jerryp@jerrypournelle.com
www.jerrypournelle.com
Copyright 2007 Jerry E. Pournelle, Ph.D.

January 2, 2007

Continuing a serious discussion on the delayed write error:

Jerry,

Last Saturday I got my very first delayed write error. The configuration was an interesting one.

Is the common factor in all of the reported instances Windows XP?

I have been running a 300GB USB drive on my Windows 2000 Pentium III system and have never seen the error.

Bob Holmes

Actually, I don't know of any common factors in getting the Delayed Write Error other than Windows, but it is mostly XP. It "usually doesn't happen." Alas, when it does, the consequences can be serious.

Hi Jerry,

I regret to say I haven't read your column in quite some while (so much tech news, only so many hours in a day!) but a friend forwarded me your piece on Delayed Write Errors because I've been suffering from a similar problem.

Have you seen this? http://support.microsoft.com/kb/908673

Regards
Dean Stacey

But you should make time for me! Thanks. Alas, the Microsoft Support article says

"This problem may occur when there is lots of traffic on the USB 2.0 hub, and you transfer files to a low-speed or a full-speed USB device. The problem does not occur when you transfer files to a high-speed USB device."

That turns out not to be the case. I have had this happen several times when backing up to perfectly good Seagate external USB drives.

A better source of information is:

Dr. Pournelle:

I suspect you've seen this,

http://www.bustrace.com/delayedwrite/index.htm

but I cannot recall seeing any mention of it in past columns. I hope this is useful in your research. It's the most comprehensive treatment of the problem that I've seen. Seasons greetings, and please keep doing stupid things so I don't have to.

Charlie Paulsen

Charles M. Paulsen Paulsen Environmental Research Ltd.

Which is as complete a discussion as I know, but it's a bit short on fixes. My conclusion after a lot of work is that Windows XP isn't reliable for writing to external devices. The Delayed Write Error can be made rare, but there's no way to be sure it won't happen. Back up to a networked drive for safety. Having said this, I will point out that I have used Ghost to do backups to an external and it has almost always worked.

Almost always. Like the Elbonian Parachutes. They usually work.

I have no data on Delayed Write Error in Vista. We can hope they have fixed it once and for all. We can hope.


Subject: December 2005 Column and Keyboard Shortcuts

Jerry,

Although this message pertains to a year-old column, the techniques and keystrokes remain relevant. While looking through the Chaos Manor Review archives the other day, I came across the December 2005 column in which you relayed the struggle to delete an obsolete log file from Overlord. Thanks for the Shift-Delete information; that's a keystroke I've wanted for a long time. You probably already knew this but deleting a file from the Command Prompt also avoids sending it to the Recycle Bin. However, I have had this fail before depending upon whether Windows thinks the file is in use.

Because my fellow programmers and I make our living at the keyboard, we rarely use the mouse. I, more than the others, use the Command Prompt quite a bit and there are three Command Prompt tricks that I use regularly, although I have no idea where I discovered them. In the following text, I'll use "C:\Program Files\OpenOffice.org 2.0\help\en\" as the example directory.

Number one is the fact that Windows allows you to "drag and drop" files onto the Command Prompt. The company that I work for uses several DOS applications in producing firmware for embedded systems and it can be cumbersome to navigate to a directory deep in the mines of Moria. I have shortcuts in my Start menu for the locations that I use routinely and each opens in the Explorer GUI. After firing up the Command Prompt, I use "c:<enter>" to switch to the appropriate drive. I then type "cd" followed by a space, move to the Explorer window, right-click (holding the button down) the desired file, and drag it over the Command Prompt window. When I release the mouse button, Windows pastes the full path and file name of the selected file onto the command line. Now, simply pressing BackSpace to delete the file name and then pressing <enter> moves the Command prompt to the desired directory. This trick gets handier the deeper into the mine you have to go.

Second up is that Windows allows the '*' wildcard to abbreviate directory names. I can move a Command Prompt from C:\ to the example directory with the command "cd pro*\op*\h*\en." One caveat is that you have to use enough letters to make the abbreviation unique. For instance, I can't begin with "cd p*\... on my computer because I have several directories at the root beginning with the letter 'p.' I don't use this one as much as I used to.

Third is the copy and paste function built into the Command Prompt's system menu (i.e. the menu accessed by the button at the far left of the title bar). The Edit submenu lets you select, with mouse or keyboard, an area of the Command Prompt window and copy it to the Clipboard. I use this to capture the text of directory listings for easy pasting into an editor when writing batch files.

Here are some keystrokes that I've also found handy, again because I don't really use the mouse. Also, I don't use the Windows key (or that other one that simulates a right-click) because I often use older keyboards. I think I found these by googling "XP keyboard shortcuts" or a similar variant. There are many more than this out there but these are the only ones that I use often enough to remember.

F2 activates the Rename feature for a file selected in an Explorer window.

Shift-F10 mimics brings up that "context menu" you see when right-clicking the mouse.

Ctrl-Esc brings up the Start menu.

Alt-SpaceBar opens a window's system menu. I use this over and over again because I can almost instantly minimize the in-focus window by quickly hitting Alt-Spacebar and then pressing 'n.'

I don't know how you really use your computer but I thought you might find some of this useful.

Brian

Thanks! I have found that learning to use the command line is well worth the effort. So are learning keyboard shortcuts. I suppose that's the very definition of "power user."


Subject: What about a chaos manor archival CD

Jerry,

Some computer magazines (e.g., Byte) occasionally release an archival CD that bundles all past issues for a certain period. These must garner sufficient sales among us Obsessive/Compulsive types who like to have full collections of everything "for future reference", but don't have the space, or can't get spouse approval. I wonder if you have ever thought of doing something similar by offering for sale a CD archive of your chaos manor site? I don't know if you'd even have to try to organize it very well, since a copy of the site as is would capture the essential (chaotic) nature of the enterprise. I know that, occasionally, I look at the older entries just to sample what was going on some years ago. It can be a nice trip down memory lane. It might also be the most durable, and lasting kind of backup you could provide for yourself!

CP, Connecticut

An interesting idea. I wonder if there would be much demand for it? It would be a good bit of work, but perhaps I can find an intern to do most of it. Thanks. Captain Morse suggests it would be a good reward for subscribers.


On Digital Rights Management

Referring to Bill Gates lobbyist power:

>> What would you like to see that clout used to get? What is a fair solution to the problem of Digital Rights? <<

Bill Gates is not our friend. Bill Gates is in favor of DRM, if you judge by his actions rather than by his recent words. Bill Gates is beginning to see what has been obvious to many of us all along; that DRM is unworkable and will come back to bite him.

As I said, I would like to see personal, non-commercial copyright infringement legalized, repeal of the DMCA (the "C" incidentally is "Copyright", not "Communications"), repudiation of the Berne Convention, retroactive rollback of copyright terms on printed works to at most seven years renewal for seven additional years, explicit legalization of any hardware or software that is needed to break DRM, and recognition that, absent a Constitutional Amendment, Congress has no power to grant copyrights at all on video and/or audio recordings.

Actually, I'm not sure what all the to-do about P2P networks is about anyway. The Supreme Court has already ruled that there is no copyright infringement absent a transfer of a physical copy of a protected work. Simply transferring bits is not copyright infringement per the Supreme Court.

Robert Bruce Thompson

That is a fairly extreme view of the Constitution and one I doubt the courts will support: copying a movie and broadcasting it has got people serious jail time. Don't rely on a libertarian view of the Constitution to assert a right to carry a camcorder into a movie showing; and I wouldn't rely on it as a defense if I happened to find a bit file of a copyrighted movie and decided to sell copies. Eric adds:

Then we'd best have that Amendment. The idea that books should enjoy greater protections than movies employing hundreds and representing investments of tens of millions of dollars is ridiculous.

Really though, this is a letter of the law vs. spirit of the law issue. Does anyone doubt that movies would have been covered if Ben Franklin had made it happen in his era rather than Edison a century later? Does the Constitution allow for copyright protection of images? If so, isn't a movie just a collection of images at 24 per second?

Eric Pobirs

Which is, I would say, the likely view of the courts. David Em adds:

This has been rehashed to death, but of course intellectual property is property. I've made any number of digital images that never existed in any material form but bits, but I've sold limited rights to them for use in books, ads, etc. The original image is still mine, as are all rights to it.

Copyright law acknowledges the difference between an image and an object. When you sell a physical painting, print, or photograph, the creator doesn't have (except in certain rare situations) any rights to what happens to that object, including participation in resale (at auction that can be orders of magnitude beyond the original price). The buyer doesn't buy the image, only the object.

Photographing a painting and using the image to sell vodka constitutes infringement. The rights to the image remain with the creator unless assigned.

David

David's The Art of David Em is alas no longer in print. I was first introduced to it at his presentation at a meeting of the American Association for the Advancement of Science; I was so impressed with his presentation that I sought out the book. For those interested in digital art, David's essay on his book can be found at this link.

Re: DRM

Doctor Pournelle

Entire industries that currently employ many people: Movies Music Books

What are all these people going to do if their livelihoods are wiped out? The current unpleasantness between the "information wants to be free" and the RIAA and MPAA takes on aspects of scorched earth or winner take all. Too much money involved. Too much politics and the power of the state involved. There is the technology djinn that no one group has complete control over. Then there is China with the factories and technology and their own idea of what intellectual property means.

It would be nice if the USA gets to have a say in this, but I suspect China and perhaps India will overwhelm the rest of the world just because they exist.

Scott Rich

Which leaves the matter unresolved. If the United States doesn't provide some kind of world leadership in this matter, others will. Protection of intellectual property is a serious matter; certainly it is for me and others who make a living through creation of intellectual works that have more than ephemeral value. Journalism produces works that earn most of their return through first use, and often the pay for first use is substantial; but is there not a need for works that take longer to craft, and have a value that remains after first publication? And shouldn't there be some protection for the artists and writers who create those works?

And then there's the entire matter of performance art: what protections should performance artists have?

This is by no means a closed matter.


A Microsoft programmer replies at length to Bob Holmes on changing default locations of MY Documents:

In the 12/18/2006 Mailbag, Bob Holmes asks and comments:

(Why does Microsoft prevent a user from changing the default location of MY Documents, Documents and Settings, et al to a drive of their choice?) This might be an opportunity for some software developer to create a program to do this.

For Windows XP and Windows 2000, Microsoft doesn't exactly "prevent" users from changing the default location of "My Documents," otherwise known as CSIDL_PERSONAL. You can do it, either during the Windows installation (using an unattended answer file) or any time after installation by manually modifying the registry. Either method requires both some arcane technical knowledge and the courage to experiment with something that just MIGHT leave your computer totally unusable. However, I've used registry technique for many years now to move my CSIDL_PERSONAL folder and many of its cousins to a drive other than the drive where I have Windows installed (I've tried to find the "real" expansion of the CSIDL acronym without success). I do that for two reasons: (1) I have several machines that dual boot using two different versions of Windows, and moving the CSIDL folders to a third drive gives me the same "My Documents" folders and files from both versions of Windows; and (2) I can safely reformat the Windows drive and clean install Windows again without losing any of my CSIDL data. NOTE: Both methods should work on Windows 9x and Windows ME, and they MAY work on Windows Vista, but I have not tried either of them on those versions...consider yourself warned.

I personally like Bob's suggestion that someone create a program to make this easier. I don't think that will happen, though...too many opportunities to change things in ways that some program somewhere won't support, creating problems for the end user and lawsuits for the developer. Example: Will the XYZ program designed to back up your "My Documents" folder find the new folder at the new location? It will if it uses the Windows API SHGetFolderPath function to get the folder path. But it won't if it "knows" that "My Documents" ALWAYS resides under the user's folder in "Documents and Settings" on the Windows drive. In that case, it will back up either the old versions of the folders that don't contain any of the user's current files, or it won't back up anything at all. XYZ qualifies as a poorly written program, and I wish I could tell you I've never seen it happen in the real world. But it does, and it will again.

The Unattended Answer File method

During Windows 2000 product cycle, I worked on the Certified for Windows 2000 Logo team, creating the test framework for third-party software developers to use to pre-test their programs before submitting them for formal logo certification testing. That logo required that applications use the Windows API SHGetFolderPath function to get the folder name/path of folders such as CSIDL_PERSONAL. To test that, we used Windows 2000 machines that had the CSIDL folders in atypical locations with atypical names. The Microsoft.com site no longer has the documentation for the Windows 2000 Logo program, but it still has the similar documentation for the Windows XP logo and that contains the same description of how to use an unattended answer file to move (and rename) the folders. You can download the framework in an executable file from this link. The document LogoTestFramework111.doc installed by the executable gives an overview of the technique in the section "'My Documents' in This Test Framework" starting on page eleven. The section "Installing Windows XP with the Unattended Answer File" on page 18 describes the procedure in detail. NOTE: the .doc file refers to a file named "unattend.txt" in a support archive on Microsoft.com. You won't find that archive on-line, but the executable creates a subfolder named \Support that contains the file.

The answer file uses two sections to create the CSIDL folders in a new location. The [GuiRunOnce] section names the users that Windows should create; the sample provided creates a user named "User1 LogoTest.1" for example. The [GuiUnattended] section creates the new "Documents and Settings" folder for all users using the command ProfilesDir="d:\Logo Test Docs & Settings" to change the name to "Logo Test Docs & Settings" and move the folder to drive D. If you follow the steps carefully, your new Windows installation will have the CSIDL folders relocated to a drive and name of your choice. WARNING: The answer file sample won't work without some modifications. You'll need to add the ProductID from your Windows installation disc, for example. And, if you don't change some of the other parameters, you'll have a Windows installation ideal for logo testing, but perhaps not well suited to other activities. A search on "unattended answer" on Microsoft.com and the internet will give you many, many more details on using answer files to install Windows.

The Registry method

Windows keeps the paths to each user's CSIDL folders in two locations in the registry: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders and HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders. If you change the path to a CSIDL folder IN BOTH LOCATIONS (for most of the folders), from that point on Windows and all well behaved applications will look for and store data using the new path. To make the changes, I suggest following these steps in order:

  1. Open regedit.exe, navigate to each key listed above and export that key to a .reg file. WARNING: If you don't know how to do that, STOP NOW and just live with the default locations; to make these changes safely, you'd have to learn more than we can teach you here.
  2. Open the .reg files in a text editor such as Notepad and copy the contents of one into the other for easier editing. You should have one line that says "Windows Registry Editor Version 5.00" and one section for each key with paths. The paths in the \User Shell Folders key section appear in hexadecimal notation, not plain text.
  3. Save this combined .reg file with a new name, and save it again with a different name so you can start over easily if you need to.
  4. In the \User Shell Folders key section, you'll find about 17 entries (different systems may have additional CSIDL paths for specific hardware/software support). Each entry starts with a name in quotes, such as "Personal", followed by "=hex(2):" and sets of two-digit hex numbers separated by commas. Regedit usually breaks these very long rows into several lines using a "\" symbol as a continuation indicator. For each row in this section, take a couple of deep breaths, do whatever you think may invoke the kindness of superior powers to watch over you...and delete everything on each row following the equals sign. You should finish with rows of names in quotes followed by an equals sign and nothing else. Save the file.
  5. Now look at the \Shell Folders section. You should see more rows than you found in the \User Shell Folders section and they should have paths in text instead of hexadecimal. For example, my \Shell Folders section has this row: "Personal"="C:\\Documents and Settings\\Noel\\My Documents" If you only want to move your CSIDL folders to a different drive, but otherwise keep the default names (recommended), you can do that by changing the "C:" to the new drive letter, perhaps "Q:". Make the changes MANUALLY one at a time. Resist the urge to do a global search/replace on "C:". Why? Because you probably have a row that says "Fonts"="C:\\WINDOWS\\Fonts" Leave that one alone...don't move it to drive Q. I also recommend that you leave "Start Menu" and "Administrative Tools" on the default drive, but if you feel brave, move them. Save again after you've changed the drive letters (and folder names if you want to do that...and if you do change the names, make sure to use two back-slashes to delimit the folder names on the path).
  6. You should find a row name in the \Shell Folders section that corresponds with each row name in the \User Shell Folders section. Copy the new path for each of those names from the \Shell Folders section and paste it into the row in the \User Shell Folders section. You will not find all the rows in \User Shell Folders that you have in \Shell Folders...expected. Save this final version of the .reg file.
  7. Copy "C:\Documents and Settings" and all its subfolders to drive Q. If you decided to change the default folder names in the .reg file, make the same changes to the folder names on drive Q. Double check your work.
  8. Think good thoughts and run your final version of the .reg file. Log off (or restart) and log back on. Open the My Documents link on the Start menu and make sure you can see everything you expect to see. Navigate to the \Shell Folders key in Regedit and verify that the folder paths you changed still point to drive Q.
  9. Log on as each remaining user on the computer whose CSIDL folders you want to relocate and repeat steps 1-8 for each of them. If you only make the changes for some users, you can remove the other users' folder copies from drive Q.
  10. I suggest that you leave the original folders in place, although you can remove the (now) duplicate files in many of them. Some applications may expect to use the AppData and Local Settings folders in their original locations. You can remove the "old" files in the default "My Documents" folder and its subfolders. Remove the data from the other folders only if you really need the disk space. You may need to put some of the data back in some folders to keep certain applications happy.

While I've used this technique successfully many times, the usual caveats apply. Microsoft does not support this technique and you take full responsibility for any bad things that may happen if you use it. If you have any doubts or trepidation about making these changes, just live with the defaults Windows uses and wait for someone to take up Bob's challenge of creating a program to do all this messy stuff for you.

Noel Nyman

Thanks! As you say, it's a bit daunting. My practice has been to have a root level folder called "Winword". Within that are dozens of folders, each with a project name, like COLUMNS, or Purgatorio, or Strategy of Technology. Those may have sub folders. I then tell Microsoft Word to look at Winword as the default location to find and store files. That makes it very easy to transfer all my documents from one machine to another, and indeed I have batch files that say "xcopy C:\winword S:\winword /e/s/d/y" which will back up all of my documents from one machine to another. (The /e/s/d/y switches tell xcopy to copy only later files, and do so in subdirectories without bothering to ask about overwriting files.)