This paper reports on some practical lessons from the evaluation of the Research on Open Educational Resources for Development (ROER4D) project, a project that is commitment to open research. The evaluation conceptual framework used is Utilization Focused Evaluation (UFE). UFE is an iterative and participatory 12-step decision-making framework for enhancing the utility and actual use of evaluations. The open approach in the project work also informed the evaluation approach used and highlighted considerations around the ethical sharing of evaluation findings and outputs, including at which stage and with whom. This paper forefronts some of the emerging benefits and drawbacks of conducting an evaluation on an open research project. Key benefits uncovered included the open and inclusive team dynamic which fitted well with the participatory UFE framework. A key challenge in this context was that evaluation work, by its nature, cannot always be conducted in the open. For other researchers and evaluators working on an open project, where outputs can be shared with a wide audience, a key takeaway is that specific care needs to be employed in the production of the evaluation outputs and making explicit what can and can’t be shared openly and at what stage. Propriety, including considerations around ethical sharing of findings, needs to be at the heart of evaluation design and implementation.
