Big Data is all the rage at the moment. Leaders in every sector — business, government, non-profit, and unions — need to start thinking about the challenges of how to manage and constructively use all the data that their operations captures.
According to IBM,
Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few.
Big Data is not just large files, it is the huge variety and vast quantity of information that modern organisations collect, whether on purpose or by accident: Call logs, website analytics, membership records, travel mileage, financial transactions and more.
Why do unions need to start to take Big Data seriously?
When it comes to organising, most organisers would like their queries to be handled faster, and to be given more accurate, more up-to-date information about the worksite or members they are visiting. When lead organisers and union leaders are thinking about industrial strategy, mapping and targeting, better decisions are made when all the facts are to hand (whether corporate reports, membership density figures, industry profiles, or property sales) in an easily understandable, usable form.
A lot of unions have access to this information — and even collect it. Membership databases keep financial records and transaction receipts of member financiality. But most of these databases were created in the 1980s or 1990s. The information stored in them is fairly limited or not easily accessible for the people who need them.
In my experience, often the decisions made about membership databases are made by the finance staff, with little consideration of the needs of organisers. All the important information that organisers need about members — levels of activism, interests, relationships, past actions or participation or previous interaction with the union — are either unavailable or kept in an Excel spreadsheet or index card.
Today, the business world makes use of sophisticated databases that bring together many data sources to build rounded profiles of customers. Large non-profit organisations like Amnesty, Greenpeace, the Red Cross or (in the USA) universities and hospitals also make use of powerful databases that integrate financial and relational information.
In the last few years, the explosion of digital marketing and social media have meant the next frontier for databases is to link this valuable social information with the heritage data that these organisations keep.
Members today have an expectation that their unions are modern organisations capable of the same levels of “customer service” that they get from large corporations. When you call your bank, utility or phone company, the call logs and other interaction you have with that company are linked to your customer record. Subsequent interactions build on that information. Comments or complaints made by you to the bank’s Facebook or Twitter are also logged; these interactions are used to generate targeted offers via email or direct mail. They are cross-matched with your credit card purchase information so that these companies have a detailed picture of your habits, consumption and behaviours.
This kind of data integration is what most Australians experience when dealing with the corporate world.
Unions need to start to match these capabilities.
However, despite all of the incredible technological advances made since the first membership databases were installed in union offices, few unions are making use of them. In fact, most unions are stuck in the “structured data” phase. The only data that can be accessed is what can be added to a pre-determined field in the database.
Most data, according to IBM, is unstructured — up to 80%. This means that all of the really useful information that could be available to your organisers or industry researchers is inaccessible. Unions are effectively operating with only 20% of the information.
The unstructured data is all the information that your union has trapped in spreadsheets, emails and Word documents. For example, consider the social media activity that your union is engaging in. Or the digital campaigning or email marketing.
Here you have access to lots of relevant information about members — their relationships and interests, their level of engagement with union campaigns.
Think about whether it would be useful for your organisers to know whether the member they were visiting regularly opened or forwarded your email newsletters, liked your union on Facebook and regularly retweeted your union’s tweets. Think about whether it would be useful to know the member’s friends, and whether they were members as well. Think about whether if would be useful to know where the member had worked previously (through their LinkedIn profile) or who they had last spoken to at the union and what they talked about. Think about whether it would be useful to know whether the member had registered to attend a union event or had bought a piece of union merchandise from your website.
Unions in the USA are already starting to do this. They’re making use of integrated databases, merging the field data collected during organising drives and Get Out The Vote efforts to better target their campaigns.
Obviously, collecting and using more data also brings challenges:
Policies related to privacy, security, intellectual property, and even liability will need to be addressed in a big data world. Organizations need not only to put the right talent and technology in place but also structure workflows and incentives to optimize the use of big data.
The next frontier for unions is to start to make use of all this unstructured data, and to turn it into actionable intelligence.
The finance tail needs to stop wagging the organising dog. Decisions about your membership database are too important to leave to your finance or accounting staff. Unions need to start to invest not just in the technology, but also in database analysts and mathematicians who can help make sense of all the information.