Breaking Down MeriTalk's sCIOal (Social) Circle Study

How good are Federal government Chief Information Officers (CIOs) at social media?

That was the question posed by media company MeriTalk in their recent study, named sCIOal Circle (get it?). According to the study (registration required), this is the first of an annual series of analyses. Thus, before the next one, it's worth looking into MeriTalk's study subjects (Federal CIOs), the metrics they used, and the conclusions they drew.

What social media metrics did MeriTalk measure?

MeriTalk chose extraordinarily basic social media metrics to measure from CIO's social media accounts, which probably says something in itself a priori. Their metrics were (see graph below, from MeriTalk study at https://meritalk.com/social-cios.php): Do you have a Facebook account? Do you have a Twitter account? Do more than 1000 people follow you on Twitter? Have you tweeted more than 10 times in the last 6 months? Do you have a LinkedIn account with more than 500 connections?

Some of this is a bit arbitrary, but it's acceptable. LinkedIn itself uses the 500 connections marker in certain ways. Having 1000+ followers on Twitter is a reasonable thing to expect if you are an active user and have professional stature; tweeting 1-2 times a month is certainly a reasonable thing to expect of someone who is "active" (personally, I would have set the bar to at least once/week).

There's one other metric which I find interesting, but who's interpretation is unclear or ambiguous. That is, the ratio of followers to people one follows. That is, if you have 10,000 followers and you follow 5,000 people, you have a ratio of 2. If you have 1000 followers and you follow 1000 people, your ratio is 1. More of my thoughts on this below.

Points were assigned as per the graphic, with a low score of 0 and a high score of 8.

What did MeriTalk conclude about Federal CIOs on social media?

Not surprisingly, MeriTalk's data showed a distribution of scores across CIOs, ranging from 0 to 8. Two CIOs, Casey Coleman (GSA) and Rick Holgate (ATF) scored perfect 8's, with two others, Linda Cureton (NASA) and Steven VanRoekel (OMB) with 7's. I don't think those four people would be a surprise to anyone who follows the Federal tech and social media space, so on some level, the MeriTalk formula is tuned to commonsense reality.

There are some interesting notes. One is that 80%+ of Federal CIOs have a LinkedIn profile. That's not surprising; I'm curious to ask the ones who don't, Why not? Another interesting note is that relatively few - seven - of them seem to use Facebook at all. That seems really low. No explanation is offered.

MeriTalk offers some additional points and analysis in their study and the accompanying press release.

Brief critique of the MeriTalk sCIOal Study

As someone who actively uses and studies social media and who has worked in and around the Federal space, I have a few notes on this study.

One is to ask, Is this important at all? Meaning: Of all the people in the Federal government you'd like to see tweeting and engaging and connecting, are Chief Information Officers high on the list? I understand that MeriTalk and some other media organizations have business reasons to be focused on these specific people, but that does not necessarily translate into an expectation that those people should be highly transparent and social. Further, without investigating whether their deputies and other staff are social, we don't get a true sense of how social their departments are. In other words, connecting to a CIO on social media might be irrelevant if they have a relatively inactive account but four key staffers tweet and blog all the time. The converse is also true; a CIO who is an 6, 7, or 8 on this scale may be social, but it may not do your business any good if you're pitching a product or a service; the key person in the food chain might be the anti-social assistant to the deputy who's reading your paperwork.

Further on looking at CIOs vs. other kinds of Federal employees: I think, for perspective, a much larger study of sub-cabinet officials would be really interesting. CIOs are important, sure, but so are a lot of other czars and chiefs and assistant secretaries and the like. My guess is that one outcome of such a study is that there are big differences between agencies in how "open" Federal employees are and how much they would tend to use social media. For example, the national security-related agency CIOs tend to score relatively low on the MeriTalk scale. Is that because of the agencies or because of the specific CIOs or both? That can be tested (i.e., if CIA, DoD, State, etc. generally score low across all officials, you could argue for agency cultural influence), but we can't conclude anything about this from the MeriTalk dataset.

A third note is that these are not identical widgets, these are people. But the human side of things isn't really taken into account. Each of the four 7's and 8's here are really interesting, dynamic people in real life, and thus, this tends to loosely translate into really interesting dynamic social media behavior. I'm not necessarily arguing that people who scored low in the MeriTalk study are uninteresting or anti-social (there are alternative explanations for, say, not using Facebook and Twitter), and I certainly don't know everyone on the list personally. But if someone is relatively anti-social, or cyber-security-concious, I don't think more knowledge of their own social media behavior will have any effect on how much they use it. Some people are just social, and some aren't, and some are in the middle, and that's okay. It's just like real life.

Finally, a note on the "F2F" ratio (followers to following). It is a real thing that can be quantified, but it's importance is certainly open to interpretation. MeriTalk judges that a low score - meaning following more people than follow you - is "better." Their #1 CIO by this metric is Shawn Kingsberry, the CIO of the Recovery Accountability and Transparency Board. Here's his Twitter account. He has a low ratio (see above graphic) because he follows 619 people, and is followed by 227 people. He seems to tweet 2-3 times a week and he's an award-winning CIO, so not to pick on Mr. Kingsberry, but I think this "ideal" ratio is completely arbitrary. For one thing, a low ratio is meant to mean that the CIO follows relatively many people, so it's easier to "get their attention," as the study states. But the ratio doesn't take into account overall numbers: Mr. Kingsberry follows 619 people and currently has a ratio of 0.37, but, for example Ms. Coleman follows 3,049 people but her ratio is a "worse" 1.66 because she has 5,069 followers. The problem with this metric is that while you can control how many people you follow, you cannot control how many people follow you. Further, it is not clear that following more people is "better" than following less; following relatively few people may be a sign that you're discriminating based on quality and actually paying attention to what high-quality people are saying.

In conclusion, the MeriTalk study was valuable on some level and whet my appetite for more measurement and analysis of what social media platforms senior Federal government officials are using, and what they are using them for. It also raises important questions about agency culture and how that can affect the social media use of an incoming senior official, particularly as the Obama adminstration transitions from its first to its second term and many sub-Cabinet and sub-sub-Cabinet appointments are expected to change hands.

Update 1: Justin Herman pointed out to me a decent commentary on NextGov that I didn't cite.