Ruka hadi maudhui makuu

We're looking for testers — try MasakhaneTool early

· 3 min read
Shamsuddeen Hassan Muhammad
Shamsuddeen Hassan Muhammad
Researcher, Masakhane / HausaNLP

We're opening up MasakhaneTool — the Masakhane community's open annotation platform — for early testing. If you annotate text or speech for African languages, work on community datasets, or teach data-collection workflows, we'd love your help shaping what we ship in v1.

What MasakhaneTool is​

MasakhaneTool is an Apache 2.0, mobile-first, offline-capable Progressive Web App for African-language NLP data collection. We built it because most existing annotation tools assume desktop browsers, fast connectivity, and Latin script — assumptions that don't hold for the African contexts we work in.

In its current shape, the tool supports:

  • Sentiment annotation (e.g., AfriSenti-style three-way labelling)
  • Named entity recognition with custom entity sets
  • LLM-evaluation workflows — score model output for accuracy, fluency, and safety
  • Inter-annotator agreement dashboards (Fleiss' κ, Krippendorff's α)
  • Virtual keyboards for non-Latin scripts (Ge'ez and others)
  • Offline-first capture with background sync when connectivity returns

Pilot deployments are running at Bayero University, Kano and Bahir Dar University ICT4D.

Why we need testers now​

We've validated the architecture and the core annotation flows internally. What we need next is real users on real tasks — annotators using the tool on their own phones and laptops, on their own networks, in their own languages — to surface the gaps we can't see from inside the team.

Specifically:

  • Usability — what feels confusing? what slows you down?
  • Scripts and keyboards — does typing in your language work the way you expect?
  • Offline behaviour — does the sync feel reliable on patchy 3G?
  • Mobile vs desktop — which workflows feel right on each?
  • Accessibility — anything that breaks for screen readers, low vision, or limited motor control?

Who we're looking for​

You don't need to be a senior researcher. We're especially interested in:

  • Annotators doing labelling work for African-language datasets (paid or volunteer)
  • Students learning NLP or linguistics
  • Researchers running annotation projects (your team becomes a pilot)
  • Educators who teach data-collection workflows
  • Native speakers of any African language — we want broad script and morphology coverage

If your work touches African-language data in any way, you fit the profile.

What testing looks like​

  • Time commitment: 1–3 hours over a 2-week window. You can spread it however you want.
  • Tasks: complete a small annotation project we provide (or use the tool on a project of your own).
  • Feedback: a 15-minute call OR a written form — your choice.
  • No NDA, no signup paperwork.
  • All testers are credited in the v1 release notes if they want to be (or stay anonymous).

How to sign up​

Three ways, pick whichever fits:

  1. Discord — say hi in #masakhane-tool and a maintainer will reach out
  2. GitHub Discussions — open a thread in the project's discussions with a brief intro
  3. Email — write to the team via the address on the About page

We'll get back within 48 hours with onboarding details and a tester invite.

If you can't test but know someone who would be great — please forward this. The strongest signal of community-built infrastructure is community-led testing.