Musings on readability assessment
I'm not one to deny the potential usefulness of computers. How could I? After working in and around the things continuously between February 1974 and November 2006 that might be a Silly Thing to do. But I did, from time to time, see people trying to do all sorts of things1 with them, not all of which struck me as particularly wise.
Including using them to assess my skill (if any) at writing readable manuals.
My first assignment...
... at the IBM Hursley development laboratory was to revise the CICS General Information manual2 for the next Release (1.6) of that venerable piece of transaction processing software. The GI manual was the only book that made any attempt to explain what CICS was for. Indeed, I'd checked it out of ICL's own Competitive Information library in March 1981 to read on the train on the way down to my job interview. I thought I had a reasonable shot at getting a job there as I was by then a pretty experienced writer. I'd been a successful technical and instructional writer (hardware and software) and programmer for nearly a decade (both employed and freelance, simultaneously). Busy chap.
Mortgage to pay. Mouths to feed.
At my very first interview of a full day in IBM Hursley (with what was then still known as "Personnel" rather than the godawful "Human Resources" it later became) I was immediately asked "Why have you decided to join IBM, Mr Mounce?" When I immediately replied "I haven't, yet" this seemed to derail the lovely chap Ron Stotter from his usual script, but we both laughed and then got on like a house on fire.
I was sent off for a coffee while they lined up my next interview. In that one, I openly admitted my (total) ignorance of CICS but said that to begin remedying this I'd read my first CICS manual that morning. "Which manual?" I pulled out my library copy. "What did you think?" [Pause] "I could barely understand it." "Why's that, Mr Mounce?" [Pause] "Well, I'm sorry, but it's badly-written, and poorly-structured."
(I have an honest streak that [still] gets me into trouble.) Mr Tact, that was me back then. I wondered if I'd now be politely shown the door...
I got the gig
The job ad I was applying for was modelled on one I'd written in ICL a couple of years earlier. Talk about poetic justice! I later learned that it was "Personnel" who had stopped "Information Development" from putting this job ad in what they correctly assumed was my local paper — they didn't know me, but they had deduced which bit of ICL I was probably working in. That rather tickled me.
My contract of employment with ICL required three month's notice from me. I didn't fancy hanging around in ICL for three months doing "make work" as soon as they knew I was off to work for the "enemy". So I did some negotiating and some date-jiggling and arranged a two-week gap between employers. I figured (wrongly!) that that would give me time to finish the current batch of freelance programming I was then in the middle of for Haymarket Publishing (but had seen little need to trouble either ICL or IBM with much [any] information3 about).
I thus marched into the IBM laboratory in June 1981. Smack bang into trouble.
Karma strikes
I was told that, far from starting work on the CICS GI manual, my first job would actually be to co-operate with a couple of academics from Birmingham University. "OK. Doing what, pray tell?" It turned out this enterprising pair had, if you please, independently taken it upon themselves to assess the poor state of that same CICS manual and wangle a contract out of IBM to re-draft the thing. And (worse!) they persuaded IBM to pay them more money than IBM was about to start paying me. That rather untickled me.
The "Birmingham" draft improved on some aspects of IBM's book — which I now had to start thinking of as "my" book, dagnabbit — but still varied alarmingly between being desperately amateurish, or inappropriately "academic", in other aspects. This (for reasons unclear to me at the time) delighted some factions4 in IBM, and horrified others.
Being perceived as some sort of unbiased neutral outside expert in the assessment of crappy manuals (!) I was gleefully handed a copy to assess. And basically told to "fix it". I may still be excused, I believe, for thinking to myself "Is this any way to run a publications outfit?" I was unaware of IBM's policy of internal development competition with itself, as it were, in the absence of any real perceived rival. So that left me sitting in the middle of a potential minefield as rival factions in IBM (not just in Hursley, either) argued the toss over the future of my little book. (I was equally unaware of just how huge a deal CICS was in IBM's software portfolio. And how bitter was the rivalry with IMS, for example.)
In early 1982...
... as I was innocently completing work on what I really, really, really hoped5 was to be my final draft... I found myself hauled into a meeting to "discuss" a "report" that had been written and circulated (though not to me) regarding a "problem" with the "reading level" of my book. This "problem" had been uncovered, not by any real-world methods I was used to (such as trying the text out on members of its target audience) but instead by using a witless computer to count up syllables, word lengths, and sentence lengths to compute and report its Flesch index. This was (supposedly) a more accurate method of assessing readability than actually sitting someone down with the book and then chatting with them, one on one, to find out how they got on with it.
Amazing.
My flabber was well and truly gasted. And there's me thinking "But IBMers are all supposed to be so smart!"
Hence the meeting with, among others, a senior programmer6 who was convinced (and, worse, had convinced an even more senior product development manager7) that this arrant nonsense would actually provide an objective and viable measure of the quality of my writing. And perhaps help make the case for product development to assume 'ownership' of the information development process to knock it into more rational shape? My information development manager was watching these proceedings, on the reasonable grounds that I was by a long way the most junior IBMer present (for all that IBM liked to maintain the fiction that it was a "single status" corporation).
I suppose it didn't help that I asked the title and rôle of each person present as few of them were people I'd had any contact with (yet they were all in this meeting because they all knew better than me how to write readable manuals). I didn't know many senior IBM managers, or programmers, or planners, or marketing folk, or finance people.
But, just as importantly, they didn't know me :-)
The senior programmer — having caused me some mild embarrassment by circulating his computerised Flesch index assessment — deserved, I felt, some pain in return. It wasn't my fault that the senior product development manager believed his programmer. I politely asked this manager (the most senior person present, I later discovered) if he would please read my choice of troublesome sections. I politely asked him each time if he was quite sure he understood them (to his visibly-growing irritation and indignation as he said "Yes" to each one). Then I apologised for having to ask him whether or not he had a PhD. He said no, he didn't. Finally, I told him that, according to the Flesch index assessment of the pieces of text he had just read, there was no way he could possibly have understood them without one.
"Don't be stupid!" quoth he.
"I rest my case" quoth I, and sat down.
The meeting broke up in considerable disorder, and barely-suppressed acrimony. I kept quiet. My job was done. I managed not to laugh. (Actually, I was hopping mad.)
I had no further...
... encounters, bruising or otherwise, with the Flesch index throughout the rest of my "career". Managers, non-writers, and programmers remained keen on it for some reason. But nobody ever again subjected my books to it. Of course, I made a couple of enemies, but that goes with the territory sometimes. Like Robert Boyd said, at some point you have to decide whether to "be" someone, or whether to "do" something. I made it crystal clear that I felt the time could have been better spent on improving the quality of the CICS product code I was documenting, but that's a whole different war story.
Hell's Teeth! Even the Common Core State Standards Initiative Text Complexity appendix is honest enough to admit (in teensy-weensy footnote #5 well-buried on page 3 of said appendix in their labyrinthine web pages) over 30 years later:
Postscript
The Readers' Comment Forms that poured in after my book's eventual publication were about 95% positive — far more positive than had previously been the case with that, or any other, book in the CICS library. But what do I know? One of my managers actually stopped passing these Forms along to me to "respond" to since basically all I was doing was writing "Thanks ever so much" on them.