Configure catalysis report

Looking at all of the graphs for your project might be overwhelming; sometimes there are thousands. You might want to reduce the number of graphs you consider at once. Perhaps you would like to start your catalysis work just by looking at data integrity, or only bar graphs, or something. You can move back and forth between this page and the next to see only the graphs you want to concentrate on right now.

Sometimes questions don't turn out to be very useful, because participants didn't understand them, or the answer possibilities you gave didn't appeal to participants, or there wasn't as much variety as you expected to find. For whatever reason, you may not want to include every question from your story form in your catalysis report.

Any question or graph type you uncheck here will not show up on the Explore patterns page. However, if you write an observation about a pattern and then hide its question or graph type, the observation will still appear in the printed report.

The graph types are described in detail on the Explore patterns help page.

If you have lots of stories, you might want to strengthen the results of your statistical tests by removing all small story subsets from comparison. The higher this threshold the stronger (and fewer) your statistical results will be. Be aware that choosing a threshold below 30 will result in test results with weaker power, which can only be considered weakly suggestive.

Changing your statistical threshold might change some test values. If you do change this value, be careful to update any observations you've already written that depend on statistical test values.

You can use a filter to limit the scope of your catalysis report to a specific subset of stories. This feature is useful when you have an important division in your story collection, such as when your stories were collected in different locations and you need to look at patterns within as well as among locations. (Some people call this "slicing" data.)

To create a filter, enter the exact short name of a question in your story form, followed by two equals signs (==), followed by either a list of exact answers (separated by two equals signs), or the low and high values of a numerical range (separated by two equals signs).

Some examples:

• Feel about == happy will include only stories for which the question "Feel about" was answered "happy."
• Location == Northern == Eastern will include stories for which the "Location" question was answered either "Northern" or "Eastern."
• Story ended == 0 == 20 will include stories for which the scale question "Story ended" was answered with a number in the range from zero to twenty (inclusive).

To negate a query, write an exclamation point just before the name of the question. For example:

• !Feel about == happy will include only stories for which the question "Feel about" was answered with any other answer than "happy."
• !Location == Northern == Eastern will include stories for which the "Location" question was anything but "Northern" or "Eastern."
• !Story ended == 0 == 20 will include stories for which the scale question "Story ended" was answered with a number outside the range from zero to twenty (inclusive).

You can filter by more than one question. Separate the two questions with two ampersands (&&). For example:

• Feel about == happy && Story ended == 0 == 20 will include stories for which the question "Feel about" was answered "happy" and the scale question "Story ended" was answered with a number in the range from zero to twenty (inclusive).
• Feel about == happy && !Feel about == sad will include stories for which the question "Feel about" was answered "happy" and the question "Feel about" (yes you can specify the same question twice) was not answered "sad."

Tips on writing filters:

• You can use these question types in a filter: boolean, checkbox, checkboxes, select, radiobuttons, slider. You cannot use these question types in a filter: label, header, text, textarea.
• For a boolean question, the answer specified in your filter must be either "yes" or "no."
• For a checkbox question, the answer must be either "true" or "false."
• For a slider, you must specify both ends of the range. Scale ranges must be inclusive.
• To filter on story length, enter "Story length == " followed by a lower and upper character limit, separated by two equals signs. You must specify both numbers. For an inclusive limit, use a square bracket. For an exclusive limit, use a parenthesis. For example, the filter "Story length == [100 == 200)" will include only stories whose text length is 100 to 199 characters. The filter "Story length == (50 == 100]" will include only stories whose text length is 51 to 100 characters. If you leave out the brackets or parentheses, inclusive limits will be assumed.
• To filter on story texts, enter "Story text == " followed by any number of exact-match texts, separated by two equals signs. For example, the filter "Story text == Office == office" will include only stories in which the word "Office" or "office" appears. Multiple texts will operate as an OR query; stories with any of the texts will be included. To create an AND query, write multiple "Story text" filters and connect them with two ampersands.
• You can't filter on the number of stories told.
• Spaces around the equals signs and ampersands are optional. They get trimmed off.
• Why two equals signs? Why two ampersands? Because you might have one equals sign or ampersand in a question name or answer.
• If you have multiple story collections in your catalysis report, all matching stories will be included, as long as the question and answer names match (exactly).
• Don't forget that you set a filter! At the top of each remaining page in the catalysis section (explore patterns, cluster interpretations, print catalysis report), just below the name of the catalysis report, you will see a reminder that you are only considering a subset of the stories in your story collection(s). The same reminder appears in the printed catalysis report.
• You can use the number of stories reported in the reminder to check that your filter is working properly. If you want to be really sure that you have only the stories you want in your report, check the graph for the question. The answers you filtered out should be missing from the graph.

You can lump choice-question answers permanently as you import your data from a CSV file. (See the Import guide to learn how to do that.) However, NarraFirma also supports display lumping -- that is, changing your catalysis graphs without changing your stored data.

Use the display lumping function with care, respect, and transparency. Never use it to distort or misrepresent what people said. Always explain why and how you used it.

To use display lumping, find the box labeled "If you want to use display lumping for choice questions, enter your lumping commands here." In that box, for each set of answers you want to lump together, on its own line, enter

• the exact short name of the question,
• two equals signs (==),
• a list of the exact answers you want to lump together, separated by two pipe characters (||), then
• two equals signs again, and finally
• the name of the new merged answer.

For example:

• If you enter Feel about == satisfied || relieved == positive, NarraFirma will count the answers "satisfied" or "relieved" to the question "Feel about" as "positive."
• If you enter Feel about == frustrated || angry == negative, NarraFirma will count the answers "frustrated" or "angry" to the question "Feel about" as "negative."

Display lumping has some limitations, as follows.

• You can only use display lumping with questions of the types checkboxes, radio buttons, and select. You cannot use it with questions of the types boolean, checkbox, slider, label, header, text, and textarea.
• Display lumping only affects the graphs drawn on the Explore patterns page and in your catalysis report. It does not affect the "Review graphs" page, which always shows your unlumped data.
• In your catalysis report, you can use display lumping with all of the questions NarraFirma generates: Eliciting question, Number of stories told, Story length, and Collection date. For example, lumping commands for collection dates might look like this:
  • Collection date == 2021-02-14 || 2021-02-15 || 2021-02-16 == Pulse 1
  • Collection date == 2021-03-05 || 2021-03-06 || 2021-03-07 == Pulse 2
  • Collection date == 2021-05-12 || 2021-05-13 || 2021-05-14 == Pulse 3
• If you have multiple story collections in your catalysis report, display lumping will apply to all of them, as long as the question and answer names match.
• However, when you are printing story cards, you cannot use display lumping for any of the generated questions. (It works for all story and participant questions, however.)

You can also use display lumping to:

• Rename an answer, usually to shorten it so it looks better on graphs. Enter the old and new answers on the same line (question == answer == new answer).
• Hide an answer whose count is very small but that cannot be meaningfully lumped together with any other answers. For example, if only 5 of 500 people said their story made them feel "weird," you can choose not to show that count on your graphs. To do this, enter (exactly) "*IGNORE*" as the new answer (question == answer == *IGNORE*). Do not leave out the two asterisks. They are in case anyone ever uses the all-caps word "IGNORE" as an actual answer.

Other tips:

• Spaces around the equals signs and pipe characters are optional. They get trimmed off.
• Why two equals signs? Why two pipe characters? Because you might have one equals sign or pipe character in a question name or answer.
• In your display lumping specification you can comment out a line by placing a semicolon (;) as its first character. That line will be ignored.
• If you make changes to your lumping scheme after you have written some observations, make sure you check to see whether your observations need to change. NarraFirma cannot tell that you have changed the way your graphs are drawn. You need to keep track of that yourself.

The catalysis process almost always requires that you choose some answers to lump together. Why? Because when you give people a list of answers to choose from, you will often find that some answers gather few responses.

Let's say that I asked people how they felt about their story, and I got a distribution of answers like this:

Every statistical test has a low-count cutoff. Many won't run if the counts to be compared are less than twenty. With this distribution, I can't run a test to detect statistical differences between positive and negative answer counts, because my "frustrated" and "angry" counts are each less than 20. However, if I combine the two negative-answer counts (frustrated and angry) together, I can create this distribution:

I can use statistical tests to compare those counts.

If your project has a lot of stories in it, you may think you can ignore lumping. But if you want to slice and dice your data into categories (by province, age, position, etc), you could still end up with low counts for some answers. To take another example, the chi-squared test (used in contingency tables) cannot be calculated if any of the table cell counts is less than five. So you can't run it on a table like this one because of the one cell that has only four stories in it.

In this case, lumping together the managers and executives, or help and respect, would improve the situation. Whether that's a good idea (should those categories be lumped together?) is one of the decisions you need to make as you go through the catalysis process.

If you have a lot of data points, or if you imported data collected with a different slider scale, you may end up in a situation where one dot on a scatterplot represents not one but several (or many) stories. In that situation, you can make scatter-plot dots semi-transparent, so a dark dot represents more stories than a pale dot.

A catalysis report contains observations, interpretations, and perspectives that refer to patterns (graphs or sets of graphs) in the story collection(s) connected to that report. Sometimes it happens that you need to change a story form or story collection after you have started working on a catalysis report. To avoid having to start the report all over again, you can copy its observations, interpretations, and perspectives to a new report.

To do this:

1. At the bottom of the "Configure catalysis report" page, with your catalysis report selected, click Export report elements to CSV.
2. Create one or more new story collections (maybe with new story forms).
3. Create a new catalysis report using those story collections.
4. At the bottom of the "Configure catalysis report" page, with your new, empty catalysis report selected, click Import report elements from CSV.
5. Choose the CSV file you saved.
6. Check over your observations. If you think the patterns might have changed, check over your imported observations to see if they still make sense.

Two important warnings:

• Always import report elements into a new, empty catalysis report. NarraFirma does not check for collisions with existing observations, interpretations, or perspectives.
• NarraFirma has no idea (and does not care) if your imported report elements make any sense with the new data you are connecting them to. If your data is different in the new story collection, it is up to you to go over each observation and see whether it still makes sense.

Notes:

• Observations are identified by their connections to patterns. Patterns are combinations of questions. Questions are identified by their short names. Thus the critical connection between two catalysis reports lies in the short names of the questions their patterns refer to. So you can transfer elements from one catalysis report to another as long as the question short names are identical.
• The question types must also be identical, because different patterns would be generated if you changed, for example, a choice question to a scale question. (Though it won't matter if a question changes from a single-choice to a multiple-choice answer; as long as the same graph types will be generated, the mapping will work.)
• It does not matter if question texts or answer lists are different in the two situations. Those things are not involved in the connections between observations and patterns. It also does not matter if you are working in a new project when you import report elements. The only thing that matters is that the question short names (and types) match up.
• If the question short names are not identical in the two reports, you can carefully edit the question short names in your exported CSV file to match those in the new situation. Be sure to change the question short names in the "Pattern" column as well as the "Question #" columns. And make sure you preserve the prefixes in front of question names:
  • S_ for story questions
  • P_ for participant questions
  • A_ for annotation questions
  Remember to save the CSV file using the UTF-8 encoding format.