Test utilities for plugins

Without test utilities, plugin authors must start the full host application and manually trigger hooks to verify their plugin's behavior. This makes the test cycle slow, fragile, and environment-dependent. Most plugin authors will skip testing entirely, producing a plugin ecosystem where bugs are discovered in production rather than in test suites. ISO 25010 maintainability.testability requires that components be verifiable in isolation; test utilities are the mechanism that makes plugin testability possible without running the full host stack.

Export a createTestHost() factory from the host package that creates an isolated test environment with mock context and hook invocation capabilities. This enables plugin authors to write fast, dependency-free unit tests.

// Exported from the host package:
export function createTestHost(options?: TestHostOptions): TestHost {
  return {
    register: (plugin) => { /* load plugin into isolated test env */ },
    invokeHook: (name, context) => { /* trigger hook and return results */ },
    getPluginState: (pluginName) => { /* inspect plugin's internal state */ },
  };
}

// Plugin author's test:
test('my plugin handles before:request', async () => {
  const host = createTestHost();
  await host.register(myPlugin);
  const result = await host.invokeHook('before:request', { url: '/test' });
  expect(result.modified).toBe(true);
});

• ID: testing-utilities

• Severity: low

• What to look for: Check whether the plugin system provides test helpers or utilities for plugin authors to test their plugins in isolation. Look for:

  1. A test host or mock host that plugins can be loaded into without starting the full application
  2. Mock implementations of the plugin context or API
  3. A createTestHost(), createMockContext(), or createPluginTestBed() function exported from the host
  4. Documentation on how to test plugins, including which test runner to use and how to set up the test environment Real-world examples:
  • VS Code: @vscode/test-electron and @vscode/test-web packages for testing extensions
  • Fastify: fastify.inject() for testing plugins without starting a server
  • WordPress: WP_UnitTestCase base class with plugin loading helpers

• Pass criteria: Count all testing utilities for plugin development. The plugin system provides at least one of: a test host/mock host, mock plugin context, or documented testing approach for plugins. Plugin authors can write tests for their plugins without running the full host application.

• Fail criteria: No test utilities. Plugin authors must start the full host application to test their plugins. No mock context, no test host, no documented testing approach.

• Skip (N/A) when: The plugin system is new (fewer than 3 plugins) and test utilities are planned. Also skip for simple config-only plugin systems where plugins provide data, not behavior.

• Detail on fail: "No test utilities for plugin authors. To test a plugin, a developer must start the full application, install the plugin, trigger the relevant hooks manually, and inspect the results. No mock PluginContext, no test host, no testing documentation."

• Remediation: Test utilities enable plugin quality. Without them, most plugin authors won't write tests, and the ones who do will create fragile integration tests that are slow and hard to maintain.

  // Exported from the host package:
  export function createTestHost(options?: TestHostOptions): TestHost {
    return {
      register: (plugin) => { /* load plugin into isolated test env */ },
      invokeHook: (name, context) => { /* trigger hook and return results */ },
      getPluginState: (pluginName) => { /* inspect plugin's internal state */ },
    };
  }
  
  // Plugin author's test:
  test('my plugin handles before:request', async () => {
    const host = createTestHost();
    await host.register(myPlugin);
    const result = await host.invokeHook('before:request', { url: '/test' });
    expect(result.modified).toBe(true);
  });