I remember in high school, and again in college, learning about the periodic table of the chemical elements.

Image of Periodic Table of the Elements from Elements Database removed at their request. The image above was created in Excel by me, so it falls under the Creative Commons license of this blog.

Russian chemistry professor Dmitri Mendeleev is credited with creating the first periodic table in 1869. He laid out the elements by atomic weight, and started a new row when the chemical properties of the elements began to repeat. Mendeleev had the foresight to leave gaps in his table where he suspected an as-yet undiscovered element should be, and he even had the temerity to make predictions about the properties of the missing elements. He occasionally switched adjacent elements if the properties did not correspond exactly to the atomic weights.

Long after Mendeleev’s time, scientists unraveled the secrets of the atomic and electronic structure of matter. It was learned that Mendeleev’s switching of some adjacent pairs of elements actually placed them in the correct atomic number sequence, defined by the number of electrons or protons in the unionized elements. The groups (columns) in the table are characterized by elements with the same valence electron structure, and thus, similar chemical properties.

The periodic table of the elements is an elegant and useful display, which arranges the elements by similar structure and properties,with characteristics changing predictably with changing rows or columns. There are sections for alkali metals, transition metals, rare earth metals, metalloids, nonmetals, and inert gases. In recent years, dozens of less periodic arrangements of objects have been forced into this format, despite the metaphor’s poor fit for most of them.

Periodic Tables of Whatever

One of the most popular is the Periodic Table of Visualization Methods. This is the faux periodic table that has led me to despise the genre. It’s about visualization, yet it’s visually unattractive. Mostly it seems that they had a set of boxes and a list of visualization methods, and pigeonholed the one into the other until they were done. And some of the examples they show (when you mouse over the “elements”) are only worthy of USA Today.

The Periodic table of awesoments is amusing. Some thought went into this PT of X. I liked Group 2 (IIA) as well as the faux metalloids, the IIIA-VIA metals, and the Rare Earths. The rest seemed to be a struggle to think of enough other elements to fill the TMs, Halides, Halogens, and Noble Gases. Perhaps I don’t share enough obscure obsessions with the author.

Most of the PT of X attempts suffer from trying to hard to force an inappropriate metaphor. And when the metaphor fits poorly, the PT of X still tries to fit into the exact same periodicity as that of the electronic shell structures of the real elements (i.e., 2-6-10-14). As a result, most are just lame grids.

You can find an amusing Table of Condiments That Periodically Go Bad, a Periodic Chart of Beer which is really no more than a list (Coors light next to Heineken next to Foster’s??), a Periodic Table of the Presidents which is simply a listing of presidents in chronological order (not by policies), a Periodic Table of Languages which lists languages in alphabetic order (not by linguistic properties), a Periodic Table of the Internet which shows no particular rhyme or reason to its organization, even a Periodic Table of Sex, which is the final proof that the wrong metaphor makes anything not fun.

One of the most creative and descriptive non-chemical periodic tables I’ve ever seen was of musical instruments, done several years ago by a former classmate of my daughter. There was a progression in size from top to bottom (e.g., violin, viola, cello, bass), which is sensible for instruments as well as for atomic elements. The groups were separated into morphologies, with strings over here, woodwinds there, the brass section, and percussion. The “Rare Earth” section was a rehash of the above instruments which had been electrified. Aside from the Rare Earths, there was no attempt to force a fit between this arbitrary X and the chemical elements. And it was an appropriate project for a seventh-grader.

Peltier Technical Services, Inc., Copyright © 2010.
 
Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
 

What’s wrong with this chart?

The first point Chad makes is that a line chart was used where an XY chart was called for. It is a shortcoming of Excel that these terms for the two types of chart were originally assigned by the first Excel programmers back in the bronze age.

This is one of the first lessons learned by first-time Excel chart makers. The little icons used in the Chart Wizard show lines with markers for line charts and markers only for XY charts, and the labels further confuse the issue by subtitling XY charts “Scatter charts”.  There is no difference in chart series formatting. Line charts and XY charts allow you to format your series the same, with or without markers, with or without lines.

The difference between these two chart types is that Line charts treat the X values as non-numeric labels, regardless of any value we humans infer from the numbers. This has to be explained many times a day on the various online forums, because people write in to ask why their charts don’t look right. At least these people are more aware than Chad’s students, who apparently don’t notice anything amiss with their charts.

I don’t mean to belittle Chad’s major point or Chad’s students. The nomenclature and icon usage in the Chart Wizard is confusing. Chad would help his students by giving them a five-minute demonstration of introductory Excel charting techniques: X in the first column, Y in the second, select the range, choose an XY chart. Actually, Chad should start by beating a little common sense into them: look at your data, look at your chart, does the chart look like it represents the data? At least you should know something’s wrong, even if you don’t know what.

The same data in an XY (or Scatter) chart at least shows a consistent and expected electromagnetic behavior.

What else is wrong with this chart?

Chad states it well:

“There’s the godawful grey background, the inexplicable pastel color scheme, and the axes running right through the middle of the plot, rendering them completely illegible.”

It’s those unsightly default formats, designed by the painters of the cave murals at Lascaux. Actually, the Lascaux artists had a better sense of color, even if they had fewer colors on their stone age palettes. I suspect the first programmers of Excel’s charting machine took the 16 original Windows colors and mixed and matched them as well as they could. Given their lack of color sense, and the inability of Windows 3.0 to satisfy even their limited color sense, the best they could do was barely tolerable. We’ve been stuck with these defaults ever since.

It’s a tedious matter to change these formats. Remove the gray background first of all. Format the axes so their labels appear in the low position. Make the plot area border medium gray or remove it altogether. Make the axes medium gray, and remove the gridlines, or make them light gray. Use bolder colors for the data points and lines.

You can then save this as a user-defined chart type, and it will be available to apply to any future charts. There are weaknesses to this approach: it is difficult to exchange these user-defined chart types between computers, the chart types replace the titles and data labels of your new chart with those of the original chart used to create the user-defined type, and sometimes the user-defined chart gallery workbook becomes corrupt, as on three of my machines.

The chart default formats have been changed in Excel 2007. At first glance they look to be improved, though I still find myself changing chart formats substantially in Excel 2007. The lines are too wide and some lines are still too dark; the colors are better but still not great. There are also several other reasons Why I don’t like Excel 2007 charts. But at least they don’t start out quite as ugly as in Excel 2003 and earlier versions.

The good news is that Excel 2007’s user-defined charts are stored in individual chart template files, which are more easily shared among users, and which seem immune to the corruption I’ve suffered with Excel 2003’s user chart gallery.

Okay, what else?

Chad doesn’t find Excel’s trendlines useful. He and his students probably have problems turning the trendline coefficients into predictions, but that can be remedied by increasing the number of displayed digits of the trendline’s number format before copying the numbers.

This is not convenient, since the numbers are in the chart, and not in the worksheet. There are means of calculating these coefficients in the worksheet, for example, the LINEST worksheet function. LINEST is a fairly powerful regression tool, and tutorials abound on the internet showing how to use it to the fullest: Extended Statistics and Polynomial Fits with LINEST, How to use Excel’s LINEST() function, and Trend Analysis With Excel to name a few, and even LINEST on the Microsoft web site.

In addition, Excel comes with an add-in, the Analysis ToolPak, which contains a number of useful analysis tools, including a regression module. Chad couldn’t find the Analysis ToolPak in Excel 2007, but it’s there, in the Analysis group at the far right hand edge of the Data tab of the ribbon. It’s possible he didn’t install this add-in; I always just perform a full installation, so I am not familiar with how to custom install the ToolPak.

Chad also doesn’t like the initial lack of axis labels in Excel 2007’s charts. These are in fact easy to apply in Excel 2003 and earlier, because the Chart Wizard includes a step where the user can easily input chart and axis titles. In their wisdom, Microsoft removed the Chart Wizard from Excel 2007, so the chart pops out before there is any opportunity to customize it. Instead the user is forced to trek through the contextual Chart Tools tabs. Hint: look for the Labels group on the Chart Tools > Layout tab.

And…?

Chad was pretty straightforward about his criticisms. I agree in principle with most of his statements. I can only argue around the fringes, and only because I have an unhealthy amount of experience with Excel.

The comments following Chad’s posts are a different story. There is much chest pounding and Microsoft bashing. As I said to a colleague, “Oh my, such anger, such invective, such inability to use Google.” One comment lamented on the poor support for Error Bars in Excel Charts. Another decried the inability to Plot an Equation in Excel.

In fairness, a good fraction of the comments pointed out workarounds or approaches similar to those cited in this article. A number suggested alternatives to Excel, both expensive commercial products and open source offerings. A few suggested that Chad enlighten his students about the secrets of Excel XY charts.

Excel is everywhere, which is both a blessing and a curse. It’s a blessing because it means there is (or was, pre-2007) a standard platform and interface that nearly everyone had on their desktop to accomplish spreadsheet-related tasks. My kids use it in school, my wife uses it, my clients use it, my dad uses it (not my mom though, but she’s an expert at Google). Excel’s omnipresence is a curse, because it leaves little incentive for Microsoft to improve. There’s no market share left to capture, aside from users of older versions, and these users upgrade mostly for nontechnical reasons.

Even when Microsoft took the opportunity to retool Excel and Office, Excel received no significant enhancements to its useful features. Excel did get a facelift, with fancy colors and themes, and more rows and columns. But the charting engine has hardly any new features, and some existing features were deprecated, broken, or otherwise compromised. The creaky statistical functions received a little attention, I guess, but they are still being criticized by the technical community. All that comes to my mind is the way the trendline formula algorithm was waylaid by an errant algorithm intended to correct rounding errors; fortunately this Excel 2007 Regression Error has been Fixed in SP1.

So, does Excel “suck” or what?

I am of mixed opinions on this question. There are problems with Excel. The statistical functions are nowhere near world class, nor are the default graphics capabilities. The new interface introduced in Excel 2007 seems like change for the sake of change; it has definitely hurt productivity greatly. The whole Office help system is pathetic; fortunately there are hundreds of third-party sourcesof help, and Google has the capability to find them.

In general, though, I think that Excel is a pretty good package, with a great deal of good characteristics, and some problems. The greatest thing about Excel is its extensibility through VBA. If you want a feature that doesn’t exist, you can write a little code (or a lot) to implement the feature. On this web site and blog I’ve shown innumerable examples of code and manual techniques to make Excel do what you want.

Excel has been good to me. I’ve used it extensively in my own work as a scientist and engineer, and more recently as an Excel developer it has paid my mortgage. A cynic might welcome the burgeoning business in upgrading existing Excel solutions to be compatible with Excel 2007, and I’ve done a number of these projects. Because of unexpected gotchas, though, I consistently underbid these jobs, and I’ve eaten more hours than I’d like. (Bidding is a tricky business. You estimate how long something will take, then double it, to compute your hours. Except in Excel 2007 projects you should double it again, and maybe a third time, to allow for the unexpected.)

Microsoft would do well to read Chad Orzel’s post and all of the similar ones they can find. Microsoft should look at the criticisms from the point of view of a regular user of your products. While Microsoft employs many heavy users of Excel, they are not the programmers and designers, but the accountants and finance people who work with numbers and dollars all day. The major users are not Ma and Pa Kettle and anyone else who they can pull in off the street to muddle through a focus group session.

What do you think? Leave your opinion in the comments section below.

Peltier Technical Services, Inc., Copyright © 2010.
 
Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
 

Rather than flame on, I’ll reproduce the comment I left under the video:Well, I sure got a good first impression of the Digital Home Video. Not very professional, but I guess it does say “Home” in the title. Did you plan what you were going to say first, or did you just have 5:00 to kill? Taking your puppy on your honeymoon, d00d?

You seem to have much more trouble with Excel and charts than someone who talks about technology. Hmm, select data, go to the Insert tab, click on a chart type, and out pops a chart. Sure, the defaults are ugly, but they’re marginally better in the latest version (2007) than before. Sure, the charting interface in 2007 is more cumbersome than before, but it’s somewhat serviceable.

And like any other object in every Office application, and a lot of non-Microsoft applications, you can right-click to find a context menu of options for changing whatever you’ve clicked on. 2007 makes it harder, because in earlier versions you could also double click to bring up a formatting dialog directly, but you can still get to the dialog through the right-click menu.

One of the strengths of Excel is how infinitely flexible the charts are. You can combine different types and apply lots of formatting to achieve effects that other packages wish they had. If you encountered any inflexibility, I’d suspect it’s a case of PEBKAC, or maybe PEBCAK. Watching the video only reinforces this diagnosis.

VBA Macro Person

I don’t know what’s more disturbing, the insulting tone, or the fact that They Don’t Get It. I haven’t moved to dot-net because in four years I’ve had exactly two potential clients ask me to develop something in VSTO. There’s no client pull. The push from my end isn’t very strong either, given the versioning and deployment issues.

 Wait until Simon sees this.

——————–

So I’m partially back in business. After getting no help over the phone support and no responses to my increasingly irate emails, I finally received an email in pidgin English, saying something like “I notice your website is suspend, hence I unsuspend from this back end. There are no more issues in the site.” The web site and blog are now up after over 24 hours in limbo. Unfortunately my email and ftp capabilities are still in the toilet, and from a business standpoint, these are more important.

I had no forewarning that anything was amiss, just suddenly the site was kaput. After I called about the suspension, the hosting company told me my account was suspended because of abuse and violation of my terms of service, so I thought maybe somebody had claimed I’d posted copyrighted material (which is all I could find in the terms of service agreement that sounded like “abuse”). Then I got an email saying it was due to a violation of the policy on number of apache connections. There was no mention of this in the agreement, other than a vague clause that referred to “overuse of the CPU”; nowhere could I find any metrics for such overuse, nor any way for me to monitor my site’s performance. Violations of the “clearly stated” and “measurable” tenets of Total Quality.

When I wondered about all of this to the tech on the phone, he said it’s a problem with a popular blog (which this isn’t really yet, but I hope it will become). That savant obviously never thought about the consequences of angering the owner of a popular blog.