Puzzler: Be in a usability study

October 7th, 2008

I get lots of e-mail from blog readers that say they learn a lot from the videos here. Well, this week we want to learn from you.

The Usability team at The MathWorks studies how people actually use MATLAB, we have a fancy room with a one-way mirror and recording equipment and everything. This gives us a chance to see where users get stuck in using MATLAB, allows us to test new features and documentation. I have been in countless usability studies over the years, and I love it because it lets me give feedback that ultimately makes MATLAB more like I want it to be.

Since not everyone can come to Boston to participate, I thought we could do the same thing over the web. If you can take a few minutes to install a screen capture software (I recommend Jing, since it is free and will upload the video easily) and attempt a short problem, that would be great.

Jing only records five minutes of video, so we have chosen a problem that can be solved in that amount of time. Think about the problem as much as you want, but record your entire screen from the moment you start using MATLAB.

Here is the problem:

What is the sum of all the natural numbers up to and including 1,000 that are multiples of 3 or 5?

The prizes:

Get a video to us by Monday October 13th 9:00am EST and you get a MATLAB t-shirt. Solving the problem is not required: we are studying the process of using the software, not the outcome. That being said, Will and I will choose special prizes for some video makers. Please “think out loud”, tell us what you like, and don’t like.

NOTE: The contest is now closed and a recap was posted. [click here]

By Doug Hull

You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

17 Responses to “Puzzler: Be in a usability study”

Stefan replied on October 7th, 2008 at 10:31 UTC :

Ah. I can’t submit a video at the time, but did anyone else get 234168?

sum(arrayfun(@(i) i*(mod(i,3)==0||mod(i,5)==0),1:1000))

dhull replied on October 7th, 2008 at 10:45 UTC :

I always find it interesting that even the simplest of problems like this are solved in such radically different ways by different people. My solution only uses one function that you used, SUM!

Too bad you can’t do a video for us, I am curious how you came to this algorithm. The process of trying and testing is interesting.

I think in the follow up for this, I will make videos explaining the techniques that each person uses. I have to admit I needed to remind myself what ARRAYFUN does.

Thanks,
Doug

Stefan replied on October 7th, 2008 at 11:07 UTC :

I’m a big fan of cellfun and arrayfun - they allow you to incorporate a FOR loop in one line. As an example, you can use this to expand it to an in-line IF statement:

oneLineIfElse = @(x,v,w)retCellElement(cellfun(@(y,z)y{(z>0)+1},{{w,v}},{x},’UniformOutput’, false),1,1);

where:
retCellElement = @(x,y,z)x{y,z};

This allows you to be more versatile in your coding by utilizing more functions handles to facilitate bulk operations. For example:

—

Search and Return - searches through the incol’th column of a cell arrayfor a numerical or textual value that matches ‘value’. It then returns the corresponding values (in cell form) in the retcol’th column.

Example: searchAndReturn(analystData, ‘Blue’, 5, [1,3]) to return the first and third columns where the 5th column matches ‘Blue’.

searchAndReturn = @(arr,value,incol,retcol)arr(nonzeros(cell2mat(arrayfun(@(row)oneLineIfElse(ischar(arr{row,incol})&&ischar(value),strcmpi(arr{row,incol},value)*row,eval(oneLineIfElse(isnumeric(arr{row,incol})&&isnumeric(value),’(arr{row,incol}==value)*row;’,'0;’))),1:size(arr,1),’UniformOutput’, false))),retcol);

—

This seems overwhelming at first, but once you start to pick it apart, you realize that searchAndReturn is the same as FLOOKUP in Microsoft Excel (i.e., look up a certain value in given column and find its corresponding value in another.)

I guess I kind of went off-topic on this one, but I’ll see if I can put together a short video explaining some of these techniques.

Joel replied on October 7th, 2008 at 11:46 UTC :

So where do we send our videos?

dhull replied on October 7th, 2008 at 11:56 UTC :

Joel,

In Jing, you are able to upload the movies. Send a link to me hull@mathworks.com, or be brave and post the URL into the comments.

Doug

dhull replied on October 7th, 2008 at 14:06 UTC :

You can also FTP the movies. Instructions here

http://www.mathworks.com/support/solutions/data/1-15WH1.html?solution=1-15WH1

Just send me an e-mail when you do with your address and t-shirt size.

Also, remember, this is a study of the software, not of you. We want to see how people use MATLAB. It really does not matter if your algorithm is longer that a posted solution. We are more interested in the process!

Thanks,
Doug

dhull replied on October 7th, 2008 at 15:08 UTC :

Everyone,

Got our first movie in. It looks good.

Along with the t-shirt, I can give you some helpful hints about your programming style once I watch it.

Thanks,
Doug

Danilo replied on October 7th, 2008 at 19:05 UTC :

There is no Jing for Linux, and I too far from the US to receive a t-shirt… so here is my solution:

I just typed

a = 1:1000; sum(a(~mod(a,3)|~mod(a,5)))

and was happy (and a bit surprised) to see it worked in the first try (resulted in 234168, just like the first comment).

Doug Schwarz replied on October 8th, 2008 at 08:43 UTC :

I think this problem is too simple to tell you anything about usability. Most people will not even have to consult the docs. For what it’s worth, I simply typed in

a = 3:3:1000;
b = 5:5:1000;
sum(a) + sum(b) - sum(intersect(a,b))

dhull replied on October 8th, 2008 at 10:02 UTC :

Doug,

You are right, that this is a very short problem. We are trying to keep the barrier to entry very low on this event, and keep things within the time limits set by Jing.

Even though there are a plethora of one-liners that solve this problem, the videos are interesting none the less. We look to see how people vary from developing at the command line vs editor, if they use shortcuts and accelerators to speed their work and so on. We can see how they have their desktop arranged, and what shortcuts they have added to their custom shortcuts bar.

There are very often small productivity gains available to people (like F5 to save and run a file vs. saving, going to the command line and then running) that are available but have remained undiscovered. Knowing what we need to make more Learnable and Discoverable is important to us.

So yes, this is a simple exercise. However, even with the limitations we have doing this remotely, there are things to learn.

Thanks,
Doug

Omid replied on October 10th, 2008 at 04:31 UTC :

Here’s the link to the video I made for this usability study
http://www.screencast.com/t/Pxnu4qqDD

Nicolas replied on October 12th, 2008 at 17:29 UTC :

I’ve uploaded my video to the ftp site, and sent it as an attachment to your e-mail. Hope it works.

Daniel Armyr replied on October 13th, 2008 at 07:39 UTC :

Oops, seems I misread the question. I calculated the sum of values multiples of 3 AND 5. No wonder I got another answer.

Oh well, video is in. Unfortunately, working while recording was a real drag because my computer became so slow. Now that I redid the exercise without recording I realize that I work slightly different when things flow a bit smoother.

Daniel Armyr

Daniel Armyr replied on October 13th, 2008 at 08:01 UTC :

Oh, and on the topic of one-liners, here is a generic one-line solution that is by far faster than anything proposed. Unfortunately, it is approximate, but the error is less than 1% for values of n > 50.

sum = 70/3*n^2;

This algoritm was found by pure trial-and-error using plot functions and sample datasets.

–DA

dhull replied on October 13th, 2008 at 10:26 UTC :

Nicolas,

That had to be the fastest solution I have seen to the problem. It was also very readable.

Everyone,

Thank you for the movies. I have gotten them all and watched them. They are being packaged up for the Usability team to watch too.

I will be making a review of the skills and different techniques used for an upcoming video.

The t-shirts will be sent this week.

Thanks for playing!
Doug

Mark replied on October 23rd, 2008 at 10:15 UTC :

This solution runs about 20x faster than the “arrayfun” solution: sum(union([0:3:1000],[0:5:1000]))

Andrew Newell replied on October 27th, 2008 at 09:33 UTC :

It runs 30x faster still if the brackets are removed:
sum(union(0:3:1000,0:5:1000))

MATLAB Central Terms of Use

NOTICE: Any content you submit to MATLAB Central, including personal information, is not subject to the protections which may be afforded information collected under other sections of The MathWorks, Inc. Web site. You are entirely responsible for all content that you upload, post, e-mail, transmit or otherwise make available via MATLAB Central. The MathWorks does not control the content posted by visitors to MATLAB Central and, does not guarantee the accuracy, integrity, or quality of such content. Under no circumstances will The MathWorks be liable in any way for any content not authored by The MathWorks, or any loss or damage of any kind incurred as a result of the use of any content posted, e-mailed, transmitted or otherwise made available via MATLAB Central. Read the complete Terms prior to use.

Doug’s MATLAB Video Tutorials

October 7th, 2008