AshEvents is an extension for the Ash Framework that brings first-class event sourcing capabilities to your applications. It simplifies the process of tracking and replaying events, making it easier to manage historical data and debug complex systems.
Why AshEvents?
State handling in applications
Most applications built today only keep track of the current state of their data. Two main reasons for this are:
Databases were designed back when computers had extreme resource constraints compared to today's standards, and storing historical data was not feasible without paying an extreme cost. Even though this is no longer the case, the practices established at that time still persist.
Building a system that is able to preserve and utilise historical data is both complex and time-consuming compared to just building a traditional CRUD-application. Unless it is a hard requirement, it often doesn't pass the cost/benefit trade-off when you want your app to be up and running yesterday.
If you never have any use for looking at historical data, then all of this is fine and you can carry on without any worries.
Unfortunately, this is usually not the case. Seasoned developers today will tell you it is crazy to develop a codebase without using Git or any other version control system, and an accountant will always record every transaction instead of just updating the current balances.
These practices reflect the idea that a domain's state is a growing set of events, with the current state derived by applying these events in order. If you want to know how you got where you are, you need to keep track of the steps you have made on your journey.
Odds are that there will be some varying degree of business value in treating your application data in the same manner.
What AshEvents brings to the table
In the Ash ecosystem, we already have the AshPaperTrail extension to help with auditing. This provides a simple way to track changes to your data. However, AshPaperTrail only tracks changes to individual records. This can lead to complex debugging scenarios if you need to look at multiple versions of records, across several resources, to figure out what happened and why.
AshEvents provides a centralised event log that records all events for your application. In addition to making system-wide auditing straightforward, the extension also provides these features:
- Event Versioning: Track schema evolution by versioning events.
- Actor Attribution: Identify who performed each action.
- Event Replay: Rebuild application state by replaying events, up to a specific point in time or event.
- Version-Specific Replay Routing: Route events to different actions based on their version, even to multiple resource actions if needed due to application refactors.
- Customisable Metadata: Attach arbitrary metadata to events for richer context.
Adding AshEvents to your app
1. Create an Event Log Resource
First, define a resource that will store your events:
defmodule MyApp.Events.Event do
use Ash.Resource,
datalayer: AshPostgres.DataLayer,
extensions: [AshEvents.EventLog]
event_log do
# Store primary key of actors running the actions (optional)
persist_actor_primary_key :user_id, MyApp.Accounts.User
persist_actor_primary_key :system_actor, MyApp.Accounts.SystemActor, attribute: :string
end
# ...
end
In order to automatically track actors, AshEvents uses resource introspection to determine the actor's primary key. This comes with the caveat that the actors you want to track must be Ash resources. Note that there is no requirement that the actors are persisted in any data layer, since there are no foreign keys/relationships created to the actors in the event log. For example, this is how MyApp.Accounts.SystemActor could be implemented:
defmodule MyApp.Accounts.SystemActor do
# No datalayer specified
use Ash.Resource,
domain: MyApp.Accounts
attributes do
attribute :name, :string, primary_key?: true, allow_nil?: false
end
end
2. Enable Event Logging on Resources
Add the AshEvents.Events extension to resources you want to track:
defmodule MyApp.Accounts.User do
use Ash.Resource,
extensions: [AshEvents.Events]
events do
event_log MyApp.Events.Event
end
# Rest of your resource definition...
attributes do
# ...
end
actions do
# ...
end
end
AshEvents will now start logging events for all create, update and destroy-actions on the User resource.
An event looks like this:
%MyApp.Events.Event{
id: "123e4567-e89b-12d3-a456-426614174000",
resource: MyApp.Accounts.User,
record_id: "8f686f8f-6c5e-4529-bc78-164979f5d686",
action: :register,
action_type: :create,
user_id: nil,
system_actor: "registration_form_handler"
data: %{
"name" => "Jane Doe",
"email" => "jane@example.com"
},
metadata: %{
"source" => "registration_form",
"request_id" => "req-abc123"
},
version: 1,
occurred_at: ~U[2024-06-15 14:30:00.123456Z]
}
If you only want to leverage AshEvents for auditing purposes, then that's it, you're done! If you also would like to enable event replay there are some more things that need to be configured and considered.
Event replay
Event replay is a powerful feature, which amongst many things enables you to:
Reset your application's state to a specific point in time, which can be a godsend during debugging scenarios.
Remodel your application domain into a new structure, by routing events to new resources.
These benefits are far from free. They require you to develop your application carefully, paying attention to these main challenges:
As your resources grow and change, legacy events will eventually become incompatible with your current resource actions.
How you deal with side-effects in your resource actions and system in general.
AshEvents provides functionality and some recommended patterns to help dealing with these pain points.
Clearing current records before replay
In order to do to an event replay, the slate has to be wiped clean by deleting all existing records from the event-tracked resources. This can be accomplished by configuring clear_records_for_replay on the event log resource like this:
defmodule MyApp.Events.Event do
use Ash.Resource,
extensions: [AshEvents.EventLog]
event_log do
# This needs to be declared in order for event replay to be enabled
clear_records_for_replay MyApp.Events.ClearAllRecords
# ...
end
# Rest of the resource definition...
end
clear_records_for_replay expects a module that implements the AshEvents.ClearRecordsForReplay behaviour, that implements the clear_records!/1 function.
defmodule MyApp.Events.ClearRecords do
@moduledoc false
use AshEvents.ClearRecordsForReplay
alias MyApp.Repo
def clear_records!(_opts) do
MyApp.Repo.delete_all("users")
:ok
end
end
Event versioning & routing
By default, all events will have their version set to 1. Once you update one of your resources in a non-backwards compatible way, you will have to instruct AshEvents to increment the version number of new events when running your resource's actions.
defmodule MyApp.Accounts.User do
use Ash.Resource,
extensions: [AshEvents.Events]
events do
event_log MyApp.Events.Event
# Any action that is not listed will default to using version 1.
current_action_versions register: 2, update: 1, delete: 1
# Actions that will handle legacy events need to be listed in ignore_actions, so that
# AshEvents won't add any event-tracking functionality to them.
ignore_actions [:legacy_register_v1]
end
# Rest of your resource definition...
end
Also, the Event resource needs to know how to process legacy events, by routing them to the appropriate actions. This is done by declaring the appropriate replay_overrides:
defmodule MyApp.Events.Event do
use Ash.Resource,
extensions: [AshEvents.EventLog]
event_log do
persist_actor_primary_key :user_id, MyApp.Accounts.User
persist_actor_primary_key :system_actor, MyApp.SystemActor, attribute: :string
end
replay_overrides do
replay_override MyApp.Accounts.User, :register do
versions [1]
route_to MyApp.Accounts.User, :legacy_register_v1
end
end
# ...
end
Application refactors
The routing provided by replay_overrides can also be leveraged if you have made significant changes to your application. Let's pretend that after some time, you decided that you wanted to separate part of the User resource into a new resource, Profile. You can then configure replay_overrides to route events to multiple action destinations in order to rebuild your application state in the new data model:
defmodule MyApp.Events.Event do
use Ash.Resource,
extensions: [AshEvents.EventLog]
event_log do
persist_actor_primary_key :user_id, MyApp.Accounts.User
persist_actor_primary_key :system_actor, MyApp.SystemActor, attribute: :string
end
replay_overrides do
replay_override MyApp.Accounts.User, :register do
versions [1]
route_to MyApp.Accounts.User, :legacy_register_v1
route_to MyApp.Accounts.Profile, :legacy_user_register_v1
end
end
# ...
end
Side-effects during replay
During event replay, all action lifecycle hooks are automatically skipped to prevent unintended side effects. This means none of the functionality contained in these hooks will be executed during replay:
before_action,after_actionandaround_actionhooksbefore_transaction,after_transactionandaround_transactionhooks
This is crucial because these hooks might perform operations like sending emails, notifications, or making external API calls that should only happen once when the action originally occurred, not when rebuilding your application state during replay.
For example, if a :register action has an after_action hook that sends a welcome email, you wouldn't want those emails sent again when replaying events to rebuild the system state.
To maintain a complete and accurate event log that can be replayed reliably, this is the recommended approach:
Encapsulate the processing of all outgoing and incoming side-effects in Ash-actions.
Ensure the resources that own those actions, are themselves event-tracked.
By containing these operations within Ash actions that are themselves event-tracked, we gain these benefits:
- They can be used inside lifecycle hooks: Since all lifecycle hooks run normally during regular action execution, if side-effects are kept inside actions on resources that are also tracking events, they will create separate events when these actions are called in for example an
after_actionhook. - All inputs/outputs become part of event data: When external API calls or other side effects are wrapped in their own actions, the inputs and responses are automatically recorded in the event log.
- Improved system transparency: The event log contains a complete record of all operations, including external interactions.
- More reliable event replay: During replay, you have access to the exact same data that was present during the original operation.
Side-effect example: Email-notifications
Here's a practical example of how to handle email notifications using an after_action hook that calls another Ash action for sending the email:
# First, define your email notification resource
defmodule MyApp.Notifications.EmailNotification do
use Ash.Resource,
extensions: [AshEvents.Events]
events do
event_log MyApp.Events.Event
end
attributes do
uuid_primary_key :id
attribute :recipient_email, :string
attribute :template, :string
attribute :data, :map
attribute :sent_at, :utc_datetime
attribute :status, :string, default: "pending"
end
actions do
create :send_email do
accept [:recipient_email, :template, :data]
change set_attribute(:sent_at, &DateTime.utc_now/0)
# This would not be triggered again during event replay, since it is in a after_action.
change after_action(fn cs, record, ctx ->
result = MyApp.EmailService.send_email(record.recipient_email, record.template, record.data)
# This will result in an event being logged for the update_status action,
# which will ensure the correct state is kept during event replay.
if result == :ok do
MyApp.Notifications.EmailNotification.update_status(record, %{status: "sent"})
else
MyApp.Notifications.EmailNotification.update_status(record, %{status: "failed"})
end
end)
end
update :update_status do
accept [:status]
end
end
end
# Then in your User resource
defmodule MyApp.Accounts.User do
use Ash.Resource,
extensions: [AshEvents.Events]
events do
event_log MyApp.Events.Event
end
attributes do
uuid_primary_key :id
attribute :email, :string
attribute :name, :string
end
actions do
create :register do
accept [:email, :name]
# After creating a user, send a welcome email
change after_action(fn cs, record, ctx ->
MyApp.Notifications.EmailNotification
|> Ash.Changeset.for_create(:send_email, %{
recipient_email: user.email,
template: "welcome_email",
data: %{user_name: user.name},
},
# You can include metadata for the email event through the changeset context
context: %{
ash_events_metadata: %{
triggered_by: "user_registered",
user_id: user.id
}
})
|> Ash.create!()
# Return the user unmodified
{:ok, record}
end)
end
end
end
With this approach:
- When a user is created, the after_action hook is triggered
- This hook calls the
send_emailaction on theEmailNotificationresource - Three separate events are recorded in your event log:
- The user creation event
- The email sending event with its own metadata
- The email update status event after getting the response from the email service
During event replay the action lifecycle hooks are skipped, so no duplicate emails will be sent, but all events will still be present in your log, giving you a complete history of what happened and a correct application state.
Summary
AshEvents is a powerful extension for the Ash Framework that simplifies event sourcing, enabling developers to track, replay, and manage application events with ease. By providing features like event versioning, actor attribution, and customisable metadata, AshEvents enhances system transparency and debugging capabilities. It also offers robust tools for event replay, allowing developers to reset application state or adapt to domain changes seamlessly.
We value your feedback and contributions! If you encounter any issues, have suggestions for improvements, or want to contribute to the project, please don't hesitate to create issues or submit pull requests on our GitHub repository.
