Describing is good: measuring is better

In a review of co-location and integration investments we conducted in the education context, we observed both successes and challenges for the players in striking the right balance of structure and flexibility; maintaining strong leadership throughout; and securing the right resources and personnel to deliver results. This experience is reflected in the literature, where the success factors fall into the following categories.

Grass roots engagement and planning: development of initiatives from the bottom-up, through extensive and ongoing consultation with community about local needs, requires follow-through to ensure the initiative responds flexibly to those needs. Success is also incumbent on the investment in partnerships and engaging the ‘right’ stakeholders – which also means the right number of stakeholders – as determined by the needs of the initiative.

Solid structures to support engagement: success in evaluation means a shared vision, commitment and clearly defined roles and responsibilities agreed between all stakeholders. A strong governance structure involving all key stakeholders, such as individuals with the skills, knowledge and traits required, and with the authority to make decisions is also important. Early attention to change management planning, as well as spending the appropriate time in order to develop strong, shared plans for the project are vital factors,  leading to agreed processes for monitoring progress and measuring success.

The right people: successful outcomes require strong and effective leadership. Leaders, managers and staff need the ‘right’ skills and personal traits, including the ability to work collaboratively, to work outside traditional sectoral boundaries, and to work creatively to identify mutually agreeable solutions to complex problems. Support and training for staff is required, recognising the fact that co-location and integration will be a new way of working for many.

Appropriate resourcing: recognition and acceptance that co-location and integration will take time to deliver results, and that investment in building community capacity to engage in the learning process, including building relationships and an understanding of the ways that community members can contribute is required. Secure resources, such as time, funding and personnel, both in the short and longer term.

“

“The rubric is designed to make sense of the conceptual and professional complexities which accompany the calls for collaboration. It offers both a developmental model of collaboration and a practical tool for individual organisations and networks to analyse their existing collaborative efforts and to plan for future success.”

Fortuitously, my co-researchers Dr Gail Winkworth and Michael White had been applying themselves to this question for some time, and we’d had conversations over the last few years about their work. Underpinning their work are two critical insights:  client need should determine the level of collaboration, and there are three ‘must have’ drivers of collaboration:

• The authority to operate – is there an authorising environment?
• The capacity to operate – can it be done – is there operational capacity to implement?
• The shared view of the public value of operating – should it be done?

Based on these insights, the Partnership Rubric has been developed as a matrix, with 18 Enabling Factors – assigned to each of the three conditions, linked to the four Types of Collaboration:

• Communicate to gain a better shared understanding of the issues confronting the target group
• Coordination to increase accessibility – especially for those who find formal services difficult to navigate
• Collaborate to address service gaps and emerging community issues, and provide more responsive ways of working, especially for highly vulnerable client groups
• Create to build civil society and create new opportunities for social, economic and civil participation.

Answering questions about capability, authority and purpose will reveal the level of complexity and therefore effort, resources, leadership, and players needed for the network or partnership to succeed. It will also reveal whether, in fact, you have a network or partnership – this is a pre-condition to the Rubric being useful.

The Partnership Rubric was developed by Gail and Michael over many projects across different organisational contexts, including Income Security, Education, Child Protection, Family Support, Juvenile Justice, Employment, Family Relationships, Family Law. Through it they have been able to locate the ‘health’ of a network, over time against these key indicators.

How does the Rubric work?

It is both a tool and an action learning cycle, specifically designed to provide networks with a feedback loop on their strengths and weaknesses. The Rubric is administered through a series of surveys, and the results are used to plan the next steps in strengthening the network in alignment with the purpose.   The questions are tailored at the commencement of the project to the language and nuances of a particular setting or sector.

As an action learning tool, it provides networks with a step-by-step ‘diagnosis’ of where they sit in relation to all the factors that influence effectiveness. Administered over time, and completed by every network member, it shows progress – or not, cutting through the anecdotal perceptions of performance, revealing the contradictions that so often lay within our self-reported confidence in the effectiveness of interagency efforts.

Rubric data can be presented graphically to assist leaders in communicating with their staff and the network itself to plan future reforms. The charts provide:

• Detailed pictures of the current status of each of the 3 enabling areas
• Comparisons with previous years achievements (when the Rubric is used repeatedly)
• Comparisons with similar organisations
• Priorities for future action.

Currently, Urbis’ Economic and Social Advisory practice is using the Rubric in two evaluations and it is providing insights into the players’ perceptions of their effectiveness as a collaborating group of diverse professionals. As an example, a recent baseline survey reflected the common over-reporting of network function – with very high levels of confidence in their effectiveness, suggesting they are operating at very high levels of collaboration.

And, when asked what are the top three dimensions that need attention to strengthen your network? ‘Know what each other does’ – which is a critical pre-condition to effective collaboration. With feedback on these results, networks can see where they are truly thriving and where some basic building blocks may need re-visiting.

The rubric is designed to make sense of the conceptual and professional complexities which accompany the calls for collaboration. It offers both a developmental model of collaboration and a practical tool for individual organisations and networks to analyse their existing collaborative efforts and to plan for future success.

Click here to download a copy of the presentation

This is an adapted version of a paper presented at the Australasian Evaluation Society (AES) conference, held in Melbourne on 9 September 2015, entitled ‘Describing is good: measuring is better – a new means of measuring the effectiveness of networks and simultaneously strengthening their function’, which Claire co-authored with Dr Gail Winkworth and Michael White.

References

Einbinder, S. D., Robertson, P. J., Garcia, A., Vuckovic, G., & Patti, R. J. (2000). Interorganizational collaboration in social service organizations: A study of the prerequisites to success. Journal of Children and Poverty, 6(2), 119-140. doi: 10.1080/713675966

Moore, M. (2004). Recognizing Public Value. Massachusetts, United States, Harvard University Press

Valentine, K., Katz, I., & Grifiths, M. (2007). Early Childhood Services: Models of integration and collaboration. Perth, Western Australia: Australia Research Alliance for Children and Youth.