Hi Mikolaj,

On 2/7/19 4:46 AM, Mikolaj Izdebski wrote:

Hello,

Today I've looked into porting Koschei from fedmsg to fedora-messaging. Unfortunately I've encountered problems that prevent me from progressing. Most importantly our RabbitMQ servers are not accessible from outsides of Fedora infrastructure VPN. Moreover, all clients must authenticate with a TLS certificate, even pure consumers that don't intend to publish any messages. For these reasons I've decided that upstream Koschei project will keep using fedmsg for consuming messages. For publishing I'm still considering fedora-messaging as authentication in this case is expected.

My main questions are: What are the plans regarding our fedmsg deployment? Are we planning to have it continue to run in parallel with fedora-messaging in foreseeable future? If not, then will at least fedora-messaging to fedmsg relay keep to be maintained?

The goal is to get rid of fedmsg within Fedora infrastructure. However, there's no deadline at the moment to migrate, so for the "foreseeable future" you're okay.

When it comes to making messages available to users outside the infrastructure VPN, we have a number of options:

1. Maintain an AMQP (fedora-messaging) to ZeroMQ PUB socket (fedmsg) bridge with no wire format change. Users outside the VPN continue to use the fedmsg library.

The downsides here are fedmsg isn't really maintained by anyone, fedora_fedmsg_meta_infrastructure also won't keep up with new message formats (not that it really does now), and ZeroMQ PUB sockets make no delivery guarantees so if missing messages is bad for your application...

The upside is clients don't have to make any changes, but what they're using is going to bitrot.

2. Maintain an AMQP to ZeroMQ PUB socket which isn't the same format as fedmsg and write a consumer API for it which is compatible with the fedora-messaging API.

The advantage of this option over 1. is that the API would be maintained and give users access to the functionality offered by fedora_fedmsg_meta_infrastructure. The same downsides of ZeroMQ PUB sockets apply, plus clients need to change, and we have to maintain more stuff.

3. Expose a consume-only AMQP broker to the public.

The advantage here is everything can use the same fedora-messaging APIs (or any AMQP client) inside or outside infrastructure. The downside is the service is exposed to the harsh world of the Internet.

Talking with Patrick before DevConf, I think this is the direction we will ultimately go, but there are a number of deployment configurations to consider so this isn't something you'll see tomorrow. I plan to do some more research and present the options in detail in the coming weeks.

Also, is there a list of advantages of fedora-messaging over fedmsg available somewhere? I'm wondering if there are any significant advantages that would justify the effort to port Koschei and other software I maintain away form fedmsg.

There is some discussion in passing[0] in the migration tools package, but you're not the first person to ask for a clear comparison with fedmsg so I think I'm going to write up a "History" section in at https://fedora-messaging.readthedocs.io/. The short version is:

* "At least once" message delivery guarantees. Publishers know when they've successfully published a message (or not). Messages will be re-delivered to consumers that fail to acknowledge messages. If fedmsg the delivery guarantee is "at most once".

* Something covered by above, but is worth noting is consumers receive messages that were published while they were offline assuming their queue has been created in the broker. fedmsg has an approximation of this by using datagrepper, but this is not reliable because of the "at most once" semantic: datagrepper misses messages, too. It's also slow.

* A focus on message schema. Consumer-breaking schema changes regularly make it to production with fedmsg, and go unnoticed for a long time. Consumers are not typically written to handle this robustly.

* Rather than a painfully fragile message signing scheme, message authenticity is enforced with the AMQP broker's authentication and authorization.

* At an API and configuration level (this is Python-only, of course), it is significantly simpler. Consumers don't need to inherit a class and implement an undocumented interface; they need only be a callable that accepts a single positional argument (the message). Topics can be defined in the configuration file or on the command line instead of in the Python code.

[0] https://fedmsg-migration-tools.readthedocs.io/en/latest/migration/overview.h...

To summarize: I understand there's no path forward for consumers outside Fedora's infrastructure and that is not nice. Until there is, the fedmsg gateway isn't going anywhere. Once there is, you'll have plenty of time to migrate, and migrating is trivial if you opt to not take advantage of a lot of the new features initially.

Regards, Jeremy