The problem with the code in these pages is that they only labeled the active chart. The problem with this is that I often have a dozen or more charts to label, and it’s tedious to select and run the code, select and run the code, select and run the code, etc. So I recently made a minor adjustment to the code that would apply last point labels to the active chart, or to all selected charts. (You can select multiple charts by holding Shift or Ctrl while clicking on them.)

It’s almost embarrassing how long it took me to enhance the code. The reason I didn’t is another “feature” of Excel 2007. In previous versions of Excel, when I wanted to carry out an action on the selected charts on a worksheet, I could check out the selected shapes, determine which shapes were chart objects, and act on each chart object.

      Dim myChart As ChartObject
      Dim myShape As Shape

      ''' SELECTED SHAPES ON THE SHEET
      For Each myShape In Selection.ShapeRange
        ''' IS SHAPE A CHART?
        On Error Resume Next
        Set myChart = ActiveSheet.ChartObjects(myShape.Name)
        If Not Err Then
          ' check for surface chart (disallowed)
          If myChart.Chart.ChartType < 83 Or myChart.Chart.ChartType > 86 Then
            ''''''''''
            '' do what needs to be done
            ''''''''''
          End If
        End If
        On Error GoTo 0
      Next
 

For some reason, Excel 2007 was unable to detect which chart objects were selected, so it would process every chart object on the sheet. Not cool if you have painstakingly added a dummy series and applied custom data labels to the chart you had not selected.

I learned that you could add a shape to the sheet, and add this shape to the selection, and Excel 2007 would recognize which chart objects were actually selected. But this is a bit more contrived than I like. Not as elegant as my usual code (ha ha).

Leave it to my friend and colleague Andy Pope to find a workaround. Andy is great at looking at things a bit sideways and seeing what the rest of us have overlooked. Visit Andy’s site for a bunch of imaginative charting examples.

Anyway, Andy realized that the selection would consist of a set of objects, and you can test whether each object in the selection is a chart object. This approach works just fine in 2007 (and in 2003!).

      Dim obj As Object

      '' check for error (i.e., selection isn't a bunch of shapes)
      On Error Resume Next
      For Each obj In Selection
        If TypeName(obj) = "ChartObject" Then
          ' check for surface chart (disallowed)
          If obj.Chart.ChartType < 83 Or obj.Chart.ChartType > 86 Then
            ''''''''''
            '' do what needs to be done
            ''''''''''
          End If
        End If
      Next
      On Error GoTo 0
 

So finally I expanded my routine. I’ve packed it into an add-in, which works just fine in Excel 2003 and 2007. Download the zip file PTSLabelLastPoint.zip and unpack the add-in (PTSLabelLastPoint.xla). Install it using the applicable protocol in Installing an Excel Add-In in Excel 2003 or in Installing an Add-In in Excel 2007.

Peltier Technical Services, Inc., Copyright © 2009.
 
Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
 

Usually the data is sorted in decreasing order of the vertical value. The chart can be filled in (above) or shown in outline only (below).

Multiple series can be stacked with unique fill colors…

… or stacked unfilled with unique line colors.

The data can even include positive and negative values.

The utility was delayed several weeks because of an unexpected incompatibility in Excel 2007. But the new workaround has been tested, and it’s ready to go.

For information, visit the PTS Cascade Chart Utility page. Details about installation and use of the utility can be found at the Cascade Chart documentation page.

The utility is priced at $29 US until Thanksgiving (Thursday, November 26, 2009). After that it will be priced at $39 US.

What makes this utility so great, you ask? The calculations were already perfectly adequate, and I didn’t change them at all. But I’ll show the new dialog, so you can see what has changed.

First, the previous interface for selecting input and output ranges was frustratingly restrictive. So I made the range selection function much more flexible. The colorful buttons on the left indicate the built-in options. In the most elementary case, all four entries (X input, Y input, X output, and Y output) are separate ranges, which can even be located on separate worksheets.

The options make range selection easier if some columns happen to be adjacent to each other, or if the same X values are to be used for both input and output. The range selection (Ref Edit) boxes and related labels update according to which option has been selected.

When the X input range has been selected, the program determines Npts, the number of points that should be used for the selected value of alpha. When the user changes alpha or Npts, the other parameter updates.

In the following case, one column serves as X input and X output (A7:A135), the adjacent column holds the Y input data (B7:B135), and the program will dump the Y output into a separate column (D7:D135).

Even this wasn’t smooth enough for me, because I got tired of scrolling up and down to select long ranges. So I made another improvement. I made the program smart enough to know that if only one row had been selected, it should use the range of data below the selected row, stopping at the first empty cell.

Couldn’t be easier.

The last change I made was to have the utility store the last LOESS parameters in each worksheet. This way, if I run a particular analysis on each sheet in a workbook, I only have to activate that sheet, run the program, and the dialog is pre-populated with the settings I need. What a time-saver!

PTS LOESS Utility Demo

Here is a demo of the new utility. This analysis only took a few minutes. I started with some Global Temperature Anomaly data from NASA.

I ran the utility to generate a smoothed temperature anomaly curve.

Then ran the utility a few more times to see the effect of alpha (number of points in the moving linear regression) on the shape of the smoothed curve. The values of 42, 25, and 12 points correspond to alpha values of 0.33, 0.2, and 0.1.

The fewer points (i.e., smaller alpha) in an analysis, the more that the smoothed curve follows local variations in the data.

Download and Install the Utility

Click this PTS LOESS Smoothing Utility link to download the utility. The download is a simple Excel add-in. Save it to any convenient directory, then install it following the instructions in Installing an Excel Add-In or Installing an Add-In in Excel 2007.

Excel includes linear regression among its built-in functions, but there is no built-in Deming Regression capability. Users who wanted this feature had to purchase third party packages, or perform their calculations in specialized statistics programs outside of Excel.

Deming Regression Utility

I have created a new utility to address this functionality gap. It is a simple Excel add-in, with a simple user interface. The calculations are based on algorithms found in the references at the end of this article.

When installed, the utility has a simple menu item on the PTS Charts menu

or button on the Add-Ins tab

The PTS Deming Regression Utility provides two methods for calculating Deming Regression parameters. You can use custom worksheet formulas, or you can use a dialog-driven interface.

Worksheet Formulas for Deming Regression

This option does not require use of the menu or ribbon buttons. The user calculates Deming Regression parameters using simple worksheet formulas.

Referring to the regression results in the previous post

The slope, intercept, and correlation calculations are each the result of a single worksheet formula:

The Deming Regression results are returned in a three-element array formula with this syntax:

=DEMING(Y-values, X-values, Horizontal)

Y-values and X-values are required arguments which reference ranges. Each range must be one row or column only, with an even number of cells, and the two ranges must be the same size. The Horizontal argument is an optional True or False indicating the orientation of the output. The default is TRUE, and the results will be output in a row.

Don’t forget, the entire range containing the array formula is selected, the formula is typed, and it is entered by holding down CTRL+SHIFT and pressing ENTER.

Here is the row output form of the analysis:

Here is the column output form of the analysis; note the FALSE argument in the third position:

The worksheet formulas are easy to use, but if you send the workbook to a colleague, they also need the utility installed to see the results. Without the utility, all calculations are replaced by #NAME? errors.

Dialog-Driven Deming Regression

The utility features a simple dialog that helps to calculate the Deming Regression results. To use the dialog, click on the Deming Regression item in the menu or ribbon tab, shown above. A simple dialog appears.

Select the appropriate ranges for the inputs and outputs, press OK, and the program fills in the results.

Calculations made using the dialog are not lost when viewed on a machine that does not have the utility installed.

Download and Install the Utility

Click this PTS Deming Regression Utility link to download the utility. The download is a simple Excel add-in. Save it to any convenient directory, then install it following the instructions in Installing an Excel Add-In or Installing an Add-In in Excel 2007.

References

Chapter 17:  Creating  Custom  Functions, Excel for Chemists: A Comprehensive Guide, 2nd Edition. by E. Joseph Billo, Copyright 2001 by John Wiley & Sons, Inc.

Incorrect Least-Squares Regression Coefficients in Method-Comparison Analysis by P. Joanne Cornbleet and Nathan Gochman, Clin. Chem. 25/3, 432-438 (1979).

The Dot Plot Utility makes quick work of Dot Plots of one or more series, such as this population pyramid replacement showing US Male and Female population distribution by age group.

Like all PTS Chart Utilities, the Dot Plotter makes regular Excel charts, so you can format them to suit your needs.

Also like all PTS Chart Utilities, the Dot Plotter works in all Windows versions of Excel between 2000 and 2007. In “Classic” Excel, a Dot Plot menu item is added to the PTS Charts menu, and this menu is added if it is not already present. In Excel 2007, this Dot Plot item is added as a button to the PTS Charts ribbon tab, and this tab is added if it does not exist. A simple dialog allows the user to select some formatting options.

The PTS Dot Plot Utility is priced at $39.00 US, but for the next four weeks it is available at a discounted $29.00. If you use this chart type frequently, you owe it to yourself to check out this timesaver.

The LOESS utility for Excel has been updated, and the interface made more flexible. The new version is described in LOESS Utility – Awesome Update, where there is a link to download the new utility.

I’ve used this utility in-house for a while, adding little enhancements here and there. I’ve come up with three main ways to use it, illustrated in the dialog screen shots below. Based on the option selected in the top of the dialog, the mode of operation is changed.

Here the data is arranged simply, with the input X and Y in two adjacent columns, and the output Y, calculated using the input X values, placed into the third adjacent column.

Here the input X and Y are in two columns, and the output Y is calculated using the input X values, but is located in another column. This is handy if you are calculating smoothed values for different values of alpha or N and placing the calculations in different columns.

Finally, the input X and Y values are in two adjacent columns, and the output X and Y are in two other adjacent columns, not adjacent to the input columns. Here the output X values need not be the same as the input X values.

When installed, the utility places a control on the PTS Charts menu. If you haven’t installed other PTS Chart Utilities, this menu will be created first.

Compare my weight over the past three years, smoothed using a seven-day moving average…

… and using the LOESS utility. The main trends are plainly visible, while the short term fluctuations have been removed.

The utility is a regular old Excel add-in, which can be downloaded in the zip file LOESS.zip (see the update below for a new version of the utility). Install this add-in using the protocol in Installing an Excel Add-In or Installing an Add-In in Excel 2007.

Try it out, and tell me what you hate about it.

Update 8 October 2009

The LOESS utility for Excel has been updated, and the interface made more flexible. It is described in LOESS Utility – Awesome Update, where there is a link to download the new utility.

Waterfall Chart with Median Indicated by Gap

The problem with the gap is that it’s hard to control its thickness closely. There is some variation from one category to the next, due I expect to rounding errors in pixel counting. If the interquartile range is small, i.e., less than twice the gap thickness, the boxes may not appear at all. Also in an earlier beta version, if the median was near zero, the boxes may appear incorrectly above or below the horizontal axis. But the decision to choose this technique was not as silly as this makes it sound.

The first waterfall chart program I came up with, about three or four years ago, used boxes that had borders. This provides a somewhat conventional appearance, and the median is obvious.

Waterfall Chart with Borders on Quartile Boxes – Excel View

I didn’t like this approach, however, because when Excel stacks column series, the border between stacked columns is twice as thick as the border around a single column. When the charts were copied and pasted as pictures, or exported as image files, this double border reverted to a single border (see below), but it made me crazy in the actual charts.

Waterfall Chart with Borders on Quartile Boxes – Export View

An intermediate version of the utility removed the borders, and used different colors for the second and third quartile boxes. The problem with this approach was deciding on two colors which do not clash (e.g., not red and green) and have approximately the same visual effect (i.e., not light gray and black). And oh yeah, the colors have to be included in the user’s color palette.

Waterfall Chart with Different Colored Quartile Boxes

So I scrapped the quartile boxes with borders and the different colored quartiles in favor of the median gaps. The gaps addressed the problems of the double thickness borders and of the different colored boxes, and if the user changed the background color of the chart, the gap was filled with the background color. Of course, everyone should want a white background.

I could have lived with the issues mentioned above (irregular gap thickness, obscuring of thin boxes), until a customer pointed out another issue with the gaps. If you convert to a logarithmic Y axis scale, the default gap is narrow at the top of the chart and wide at the bottom. The following chart exaggerates the effect.

Logarithmically Scaled Waterfall Chart with Median Indicated by Gap

Since I now can’t live with the median gap approach, I need to select another way to display the median. Another way to show the median is with a horizontal line, which I thought I had tried, but I can find no residue of such trials. You can use a dark line or a light line.

Waterfall Chart with Median Indicated by Dark Line

Waterfall Chart with Median Indicated by Light Line

The light line looks like the gap, except the pixel counting for a line produces consistent line thicknesses, at least until Excel 2007, where antialiasing runs amok. But I think I prefer the dark line.

I could combine the horizontal line with a bordered box that spans both interior quartiles, giving the appearance of the bordered quartiles without a double border at the median.

To help answer this question, I’m asking my readers:

What technique should the Box and Whisker Chart Utility use to indicate the median value? Should more than one option be offered?

• Gap between Quartile Boxes
• Border on Quartile Boxes
• Differently Colored Quartile Boxes
• Dark Line at Median
• Light Line at Median
• Dark Line and Dark Interior Quartile Box
• Something Else — Please Describe

Any of these alternatives will be fairly quick to implement, and any customers will have the option to upgrade for only the price of an email.

Update 27 June 2009

Sometime late in May or early in June, I decided to switch to a discrete line to encode the median, and made the needed adjustments to my utility. This is the next-to-last option shown above. Using this approach actually simplified the utility.

A peek at the Chart Tamer dialogs shows how it accomplishes its goals. First, Excel’s available chart types have been whittled down by eliminating pie and donut charts, radar charts, and all 3D charts. Based on the relationships which you wish to display (more than one can be selected for each chart), and on how you wish to feature this information, Chart Tamer offers a subset of these for you to select from.

For example, if I want to display some time series data, Chart Tamer disables chart types which are unsuited for this data.

The resulting chart is clean, without the extraneous elements of a default Excel chart. By default the chart includes the legend, which I’ve deleted in the sample below. I’ve also enlarged the chart within its enclosing shape, to reduce the expansive borders typical of Excel charts prior to 2007.

I’ve left out the dialog that allows you to show chart and axis titles and gridlines.

Chart Tamer cleans up charts by employing color schemes designed by an expert in human visual physiology. Colors can be defined for the data elements of the chart . . .

. . . and for non data elements as well.

Chart Tamer can be used to clean up existing charts using the settings for new charts. Chart Tamer includes a color picker to help you format cell text and background colors, and it includes a cell border formatting feature (below).

One issue to be aware of is that Chart Tamer changes the default palette in your Excel 2003 installation, replacing those awful default Excel colors with its own eyesight-friendly shades. In general this is a good thing, unless you’ve customized your palette to match your company logo or favorite sports team’s uniforms.

I tested the beta version of Chart Tamer, but haven’t done anything more than drive this version quickly around the block. The beta operated smoothly, and didn’t break anything; so far the final version looks good as well.

BonaVista Systems offers a 30-day trial of Chart Tamer. They didn’t pay me anything to write this review, they didn’t even ask. I just thought it was something that my readers might be interested in.

In order to reduce the tedium of manually rearranging the data and then manually constructing the chart, I’ve developed the PTS Merimekko Chart Utility. Like all of my utilities, it reduces the manual process to simply selecting a range, choosing a couple of options, and admiring the result.

There are some problems with Marimekko Charts. Some of the data in the “mekko” is encoded as linear dimensions, that is, the widths and heights of the rectangular shapes comprising the chart. The linear comparisons suffer from the shapes being stacked: without a common baseline it is more difficult to compare their dimensions.  Worse, some of the data is encoded as the product of the linear dimensions, that is, by the area of the shapes. Because of this, it has been said that the Marimekko is hardly better than a square pie chart.

These issues can be overcome by replacing the Marimekko with a set of bar charts, perhaps arranged in a panel chart.

Despite the shortcomings of the Marimekko, it is broadly used in many industries, particularly in portfolio analysis, such as used in marketing departments and in financial institutions. The Marimekko can provide a rapid qualitative view of the constituents of a portfolio.

I’m working on some other utilities. One major one, which will probably start out as a bunch of separate parts, is an enhanced charting user interface for Classic Excel and Excel 2007. This was inspired by my supreme frustration trying to work with the Excel 2007 charting interface, which was one step forward and twelve steps back. The overall plan takes about 80% of the Excel 2003 interface, 10% of the Excel 2007 interface, and 10% of stuff I can do better than either. The Format Error Bars utility above is part of this, and in the works are Format Axis, Format Series, and Format Chart Titles dialogs. These all take the bare necessities of formatting, without all the eye candy introduced in Excel 2007, thus reducing the necessary tabs back to one or two. I have an improved Select Source Data dialog, which uses two tabs as in Excel 2003: the Data Range tab is now nearly functional, while the Series Data tab is still problematic.

Eventually I plan a Chart Advisory Wizard, which will take the best of the familiar Excel 2003 Chart Wizard, throw out all the useless chart types (and you know what they are), and add a few other tricks. For example, explicitly stating which row(s) and column(s) are to be used for series names and category labels, and selecting a range of cells on which to position the chart. Both of these features were part of an ancient version of Excel, and were deprecated in Excel 97 or even earlier. I think it’s time to reprecate them, and invent new words while I’m at it. Since the word “Advisory” is in its title, this utility will include some guidelines to help users make useful charts. This might be through an expert system dialog, or through some kind of algorithm based on the selected data.

After reading dozens of complaints from people who are now switching to Excel 2007, I’ve started working on a Point Mover tool, which will bring back the ability to drag a point to change the underlying data. This is much harder than it seems, because if the cell with the data for a point contains a formula, you need to invoke Excel’s Goal Seek feature to adjust the precedent cell indicated by the user, and if this precedent cell also contains a formula, well, you have to trace the precedent chain back an indeterminate number of steps.

Another tool which I’ve adapted to several projects is a Chart Zoomer that allows the user to select a rectangular section of a chart, and somehow process the selection. I must say, this is one of the coolest things I’ve done in Excel, and it took many nights and weekends to get working. Among the actions that can be taken on the selected rectangle are adjusting the axis scales so this region fills up the plot area, and selecting the points within the rectangular regions for subsequent statistical analysis. The difficult part of this routine, well, now that the whole mouse selecting piece is in place, is making the utility flexible enough that users can do whatever they want once they’ve selected the rectangular region.

I hesitate to ask, since I’m already running dangerously low on “free time”, but are there any tricks I haven’t mentioned that would make a kick-ass utility?