Live gRPC Server Test Fixture

This was my equivalent of the Django Live Server Test Case but for Python gRPC services.

The only part I am currently unhappy with is the way in which the fixture identifies the functions it needs to monkeypatch. The identification itself might not be needed at all as the currently generated gRPC servicer classes only expose the GRPC functions so you could just monkeypatch them all.

from concurrent import futures
from dataclasses import dataclass

import grpc
import pytest

from rpc.conf import get_config
from rpc.handlers import RPCHandler
from rpc.test import as_test_handler


@dataclass
class TestingServer:
    server: grpc.Server
    channel: grpc.Channel
    port: int


@pytest.fixture
def live_grpc_server(monkeypatch):
    config = get_config()
    with futures.ThreadPoolExecutor(max_workers=1) as executor:
        server = grpc.server(executor)
        for servicer_config in config.service_configs:
            servicer = servicer_config.servicer
            for name, func in servicer.__dict__.items():
                if hasattr(func, "__wrapped__") and issubclass(func.__wrapped__, RPCHandler):
                    monkeypatch.setattr(servicer, name, as_test_handler(func.__wrapped__))
            servicer_config.add_to_server(servicer, server)
        port = server.add_insecure_port("127.0.0.1:0")
        server.start()
        try:
            with grpc.insecure_channel(f"localhost:{port}") as channel:
                yield TestingServer(server=server, channel=channel, port=port)
        finally:
            server.stop(None)

This could then be used as follows:

@pytest.mark.django_db
def test_live_server_cannot_authenticate_with_a_made_up_api_key(live_grpc_server):
    client = user_auth_grpc.UserAuthenticationStub(live_grpc_server.channel)
    request = user_authentication.AuthenticateAPIKeyUserRequest(
        api_key=user_types.APIKey(value="not on your nelly"),
        device_profile=user_types.DeviceProfile(
            device_id="137c4fa9-82a0-4360-b651-ba1e5b5a63c0",
            display_name="the unlikely visitor"
        ),
    )

    with pytest.raises(grpc.RpcError) as execinfo:
        client.AuthenticateAPIKeyUser(request)
    assert execinfo.value.code() == grpc.StatusCode.UNAUTHENTICATED

To me this approach felt like idiomatic Django testing (with pytest), but was felt to be a little too magic.

In this instance my understand is that magic refers to the manner in which the fixture hides away much of the details of the gRPC API itself (not our api, that of the gRPC library). Creating a client and a server and connecting them together is a fairly complicated exercise and by moving it all behind a fixture denies the person writing the test the opportunity to really see how it all does fit together.

Whilst I do believe I understand the point, I don't think I agree in this case that exposing more of the underlying plumbing is a desired thing.

Exposing how one creates a gRPC server with the grpc library directly is not all that helpful as it is already hidden away within the rpc package in mos-sdk-py for the simple reason that it is non-trivial and easy to get wrong. There is no need to re-invent the wheel every time you need a gRPC server.

Yes, having the fixture return the instance of TestingServer is an option. It would then be the testers responsibility to create the client and connect it to the live testing server. This is something a consuming developer is likely to need to do many times and is not currently abstracted away in any manner (perhaps it should be with a client registry in the rpc package?). So I can see some value in exposing it. It just feels less ergonomic for the consumer. Perhaps a compromise? The TestingServer is what is yielded but a second fixture, or even just a regular function could be used to create the client. Still feels a little unnecessary.