Do your tech choices empower a test and learn cycle?

twitter:title

Test, learn and iterate is a core mantra of the product mindset. Technology needs to enable, not block this. 

description

twitter:description

https://www.datocms-assets.com/55475/1710899345-marielle-ursua-31w2muz6ydg-unsplash.jpg?auto=format&fit=max&w=1200

Customers providing feedback on a product

twitter:image

twitter:image:alt

twitter:card

twitter:site

Test, learn and iterate is a core mantra of the product mindset. Technology needs to enable, not block this. Selecting the right tools provide this flexibility both now, and into the future.

Diligent Product Managers test every aspect of their product - early and often. Customer interviews, light-weight prototyping tools, running beta programs. We&#39;ll do anything we can to gather customer feedback. We need to ensure we are solving the right customer problem, in the right way.

A challenge in this process is that some aspects of a product can only be truly tested in production. This means live code, in the hands of paying customers.

At this point, the cycle time of the test and learn feedback loop can slow dramatically. Which is not all bad. Releases that lead to production outages are a bad thing after all. (Speeding up releases cycles with Continuous Delivery practices is a different topic.)

The real headaches for Product Managers begin when we can&#39;t respond to customer feedback in a timely way. Whether is be about a bug or a new feature. Customers don’t care about herculean challenges for the development team. They only care about whether the product is helping them get their job done. And they are rarely shy in letting you know about it.

Product Managers thrive on actionable customer feedback about feature or process improvements. We get frustrated when told by a tech lead that it’s not that simple and it’ll will take weeks to change. We die a little bit inside when told its because of a technical architecture decision.

I’ve seen this all too often. Traditional tech stacks that are rigid and inflexible. They have business logic baked into the architecture. They can make a simple change to a business rule take multiple sprints to implement.

The good news is that software is rapidly evolving. It&#39;s getting better at providing business agility every year. Domain Specific Languages (DSLs), modular toolchains, separation between back-end databases and business logic layers. These are some of the emerging trends that are reshaping the way software is built. The more I learn about these the more excited I’m becoming about the future of product development. They’ll mean more time spent on building the differentiators, not the table stakes.

It doesn’t matter if you’re build a new greenfield project, or replacing a legacy system. Providing flexibility and adaptability to the business is critical. Technology should be a business enabler, not a blocker. Make sure your technology choices support this. The success of your products depends on it.

Test, learn and iterate is a core mantra of the product mindset. Technology needs to enable, not block this. Selecting the right tools provide this flexibility both now, and into the future. Software tools and processes need to fuel positive feedback loops.

Ash Framework is ~4 years old, and we’ve only just now introduced the ability to generate resources, or new Elixir projects with Ash installed using a CLI. We’ve added these thanks to our latest project <a href="https://hexdocs.pm/igniter">Igniter</a>! Before I talk about our new generators, however, I’d like to talk about why I avoided them for so long.

<h2>Problems with Generators</h2>

<h3>Generators are a slippery slope</h3>

When a library begins providing code generation, a common design problem is leaning too heavily on these generators. Often times, instead of changing how the underlying tool works, the generators will be modified and you end up with a sort of “best-practices-drift”. The opinions of the maintainers are partially embedded into the bits that you generate, and partially embedded into the underlying library code. 

To have a successful relationship with code generators and your library , you must ensure that generators do not introduce new or different opinions. If there is something that a framework/library author thinks that users should be doing by default, that opinion must be embedded into the library directly. For example, lets say that you’ve got a web framework with a toggle that allows SSL to be turned on or off. Instead of updating your generator/installer to set that option to <code>true</code> by default because you think that is the right default behavior, make the option default to <code>true</code> if its not specified.

Another factor here is that we want our framework code to be as concise as possible. In Ash, defining a resource with hand-written code and generating it with our new <code>mix ash.gen.resource</code> command take almost the same number of keystrokes. When you have generators, the incentives go the other way. It becomes much easier to add something to generated code than it is to remove a piece of required boilerplate.

<h3>Configuration files become stale</h3>

When working users initially set up <code>credo</code> , they will typically run <code>mix credo.gen.config</code> to generate a configuration file. This configuration file contains all of the checks that credo knows about when the config was generated. What this means is that when new Credo checks are created added by its maintainers, there is no reasonable way to automatically include them in your config file without monitoring how the project is changed, what the new modules are, etc. 

A critical observation here is that there is no elegant way to do this without the introduction of a new set of tools. This isn’t a shot at Credo, or any other library/framework that does code generation this way. This is an example of something that requires a tool like Igniter.

<h3>Security updates &amp; bug fixes require manual resolution</h3>

The de facto authentication solution for Phoenix is <code>mix phx.gen.auth</code>. The way this works is that it generates an entire authentication system for you, directly into your Phoenix application. If a meaningful change, how would people find out about what changed and why? If someone were to discover a security vulnerability in that generated code, what would we, as a community, do? 

We can’t say “go update to the latest version of Phoenix” and you’re good. It will be up to you to figure out how to mitigate the security issue in your now-customized authentication code. This is an example of something that cannot be solved by Igniter.

We solve this in AshAuthentication by providing code that we manage on your behalf. There are trade-offs here of course, but this strategy means that if some security vulnerability is discovered in AshAuthentication, getting a fix is as simple as <code>mix deps.update ash_authentication</code>.

<h3>Installers are one way roads</h3>

Many developers, even experienced developers, have trouble with the decision point at the outset of a project. Do I use <code>mix phx.new</code>? Do I use <code>mix new</code> and add Phoenix? What options do I pass to <code>phx.new</code>? 

Here are some examples of options you might want to add when creating a new phoenix project with <code>mix phx.new</code>. <code>--no-assets</code>, <code>--no-dashboard</code>, <code>--no-ecto</code>, <code>--no-esbuild</code> , <code>--no-gettext</code>, <code>--no-html</code>, <code>--no-live</code>, <code>--no-mailer</code>, <code>--no-tailwind</code> . For experienced Phoenix developers, maybe you know which of these you want and when. For beginners, they have two choices:

<ul>
<li>Learn everything you need to know to know about how Phoenix works to understand which parts you don’t need.</li>
<li>Opt in to everything, regardless of whether or not they need it, dealing with a potentially more complex piece of machinery than they actually needed at the outset.</li>
</ul>

In my opinion, the journey should go the exact other way around. <code>mix phx.new</code> should get you the most basic version of a Phoenix project. Then when you want Ecto, or Tailwind, or ESbuild, gettext, you run a command to add that to your application.

<h2>Rethinking code generation with project patching</h2>

There is a better way to approach this entire problem. It requires a new kind of tool and a new way of thinking. 🏗️ <code>Igniter</code> is the tool, and it operates under this new way of thinking. What are the new rules?

<ol>
<li>We think not in terms of “generating files”, but in terms of “patching a project”. This might mean “creating or modifying modules”, or “adding configuration”, or “adding dependencies”. </li>
<li>We don’t write things that modify text, we write things that modify ASTs (Abstract Syntax Trees). We modify code, not files. No regular expressions or string operations. Instead, we use <a href="https://hexdocs.pm/sourceror/readme.html">Sourceror</a> to modify AST.</li>
<li>The source code is the source of truth, not the compiled code. For example, if we want to know if a module exists, we don’t check with <code>Code.ensure_compiled?/1</code>, we search the source code for that module with <code>Igniter.Code.Module.find_module/2</code>.</li>
<li>We acknowledge that project patching is a best effort transformation, not a perfect science. We must present information to the user explaining any changes that couldn’t be made, allowing them to manually intervene to make those changes.</li>
</ol>

<h2>What does it look like for library authors?</h2>

We write a type of composable mix task called an <code>Igniter.Mix.Task</code>. Here is a very simple one, taken from the <code>spark</code> library:

<pre><code class="elixir">defmodule Mix.Tasks.Spark.Install do
 @moduledoc &quot;Installs spark by adding the `Spark.Formatter` plugin, and providing a basic configuration for it in `config.exs`.&quot;
 @shortdoc @moduledoc
 use Igniter.Mix.Task

 def igniter(igniter, _argv) do
 igniter
 |&gt; Igniter.Project.Formatter.add_formatter_plugin(Spark.Formatter)
 |&gt; Igniter.Project.Config.configure(
 &quot;config.exs&quot;,
 :spark,
 [:formatter, :remove_parens?],
 true,
 updater: &amp;{:ok, &amp;1}
 )
 end
end
</code></pre>

<h3>Builder Pattern</h3>

All of the work happens in <code>igniter/2</code>. Notice that the first argument is an <code>igniter</code>, which we modify and return. This is called the builder-pattern. If you’ve used <code>Plug</code>, or <code>Ash.Changeset</code> or <code>Ecto.Changeset</code> you will be familiar with the concept.

<h3>High level project API</h3>

Another thing to note, is that we provide pre-built useful project patchers out of the box in igniter. For example, <code>Igniter.Project.Formatter.dd_formatter_plugin/2</code>, which adds a plugin to the <code>.formatter.exs</code>. It is <code>Igniter</code>&#39;s job to deal with any complexities of how the user may have written a <code>.formatter.exs</code> file, and to show them a message if we can’t apply the patch.

This same thing is true of <code>Igniter.Project.Config.configure/6</code>. You tell <code>Igniter</code>, “this specific config should have this value”, and it handles the rest!

<h3>Low level code API</h3>

Not pictured in that task, is that <code>Igniter</code> also has a low level API for modifying source code. This is critical when a project wants to do some kind of patching unique to that library. We encourage libraries to implement custom high level APIs that use these low level APIs. For example, when using <code>mix ash.extend Resource ExtensionName</code> , we call a function provided by <code>spark</code> :

<pre><code class="elixir">Spark.Igniter.add_extension(igniter, Resource, Ash.Resource, :extensions, ExtensionName)
</code></pre>

I will show an abridged version of <code>Spark.Igniter.add_extension</code> here, so you can get an idea of it.

<pre><code class="elixir">def add_extension(igniter, module, type, key, extension) do
 Igniter.Code.Module.find_and_update_module!(igniter, module, fn zipper -&gt;
 case Igniter.Code.Module.move_to_use(zipper, type) do
 {:ok, zipper} -&gt;
 Igniter.Code.Function.update_nth_argument(zipper, 1, fn zipper -&gt;
 Igniter.Code.Keyword.put_in_keyword(zipper, [key], [extension], fn zipper -&gt;
 Igniter.Code.List.prepend_new_to_list(
 zipper,
 extension
 )
 end)
 end)

 _ -&gt;
 {:ok, zipper}
 end
 end)
end
</code></pre>

Spark.Igniter.addextension/5 (abridged)

The workflow looks something like this:

<ul>
<li>move to the code <code>use Ash.Resource</code></li>
<li>move to the argument at index one, i.e <code>use Ash.Resource, opts</code> ← here</li>
<li>set the <code>:extensions</code> key to <code>[extension]</code>, and if its already present, set it to <code>[extension | other_extensions]</code></li>
</ul>

Notice how we use tools to traverse source code. We are using a data structure called a <a href="https://hexdocs.pm/sourceror/Sourceror.Zipper.html">Zipper</a>. Notice how even in this case, we aren’t doing any “parsing” or string operations. We’re moving around through code. This is an example of applying rule #2. By operating on the source code and not the text, we can make semantic changes. This results in a better user experience, and in the tooling being able to help you in surprisingly delightful ways.

For example, in step 3 above, if the extension we’re adding has been <code>alias</code>ed above, <code>Igniter</code> can still determine equality in <code>prepend_new_to_list</code>. It will also use existing aliases. For example, lets say you had a patcher for updating the list found in <code>@list</code> below.

<pre><code class="elixir">defmodule Foo do
 alias Foo.Bar

 @list [Bar]
end
</code></pre>

If you did <code>Igniter.prepend_new_to_list(zipper, Foo.Bar)</code>, the resulting list would be untouched, because we know that <code>Bar</code> is actually <code>Foo.Bar</code> with an alias. Try writing that regex and see how you do. 😆

Additionally, if you did <code>Igniter.prepend_new_to_list(zipper, Foo.Bar.Baz)</code>, the resulting list would be <code>@list [Bar, Bar.Baz]</code>, because we determined that <code>Foo.Bar</code> was already aliased.

There is one last bit of glue that really enhances the user experience:

<h3><code>mix igniter.install</code></h3>

<code>mix igniter.install package_mame</code> will actually check for an appropriate version dependency from <code>hex</code>, and then will add it to your <code>mix.exs</code> file automatically.

Then, we look for an <code>Igniter.Mix.Task</code> defined in that package called <code>mix package_name.install</code>. If it exists, we run it! It is a very simple concept, but it makes a huge impact on the developer experience. Lets take a look at what the new developer experience actually looks like.

<h2>Developer experience with Igniter</h2>

Lets say I’m starting a new project. I know that part of my project will include a DSL for defining workflows. I’ve got nothing so all I know is that I need a new Elixir project and that <code>spark</code> needs to be present. Lets make a new project:

<code>mix igniter.new my_app &amp;&amp; cd my_app</code> 

This is a thin wrapper around <code>mix new</code> that installs Igniter immediately. You can also install packages while creating a new project, or tell it to use <code>mix phx.new</code> (or whatever other installer you want), which is great for giving users a single command that starts a new project with your tool set up. For now, though, we’ll just use <code>mix igniter.new</code>

Now lets install Spark

<code>mix igniter.install spark</code>

First, it tells me that there are some dependencies I need to fetch:

<img src="https://www.datocms-assets.com/55475/1721103860-cleanshot-2024-07-11-at-12-54-01.png" alt="">

I accept, and I’m presented with an explanation of the changes it would like to make to install spark.

<img src="https://www.datocms-assets.com/55475/1721103913-cleanshot-2024-07-11-at-12-54-47.png" alt="">

Next I realize that I want an API to call the workflows I’ll be defining. As seen above, <code>mix igniter.install ash</code> will take me through the whole process.

Oh, you know what? I need to persist the results of these workflows in a database! <code>mix igniter.install ash_postgres</code>. Its just that simple!

Oh, now I realize I want to build an admin dashboard with Phoenix LiveView. What do I do? Thats right, just <code>mix igniter.install phoenix_live_view</code>...uh oh. All that does is add the dependency to my <code>mix.exs</code> file. Where is my fancy installation process?? There is no <code>Igniter.Mix.Task</code> called <code>phoenix_live_view.install</code> 😢

And this is the catch with generators that aren’t written as project patches. All of the initial structure that you get from <code>mix phx.new</code> can only be gotten with <code>mix phx.new</code>. There is no process to add Phoenix to an existing application (that I know of). This is where it comes down to the community. 

<h2>Where do we go from here?</h2>

The more our projects are written to be additive to a project even if all that we do when we see a conflict between a file we wanted to create and one that is already there is say “hey I wanted to create this file, but you already did, so I couldn’t make that change”, the better. This means that project patching doesn’t have to be a mega fancy process, it just has to be written in such a way that it expects that the project already exists. Thats where the “best effort” rule comes into play.

As for Ash Framework, all of our projects will soon have an igniter installation setup, and any existing generators will be migrated to use igniter as well. Igniter will be the basis for any code generation, installers, upgrade scripts, etc.

I know that others are very interested in this tooling, and I hope that we can extend this DX to as many packages in the Elixir ecosystem (that need it) as possible. I also welcome others to help improve and expand Igniter over time! It takes a village, and if I get my way, Igniter will soon be a ubiquitous tool across the Elixir ecosystem.

I hope you are all as excited about Igniter as I am! 

Happy Hacking 😎

Ash Framework is ~4 years old, and we’ve only *just now* introduced generators and installers. As of Ash 3.1, we’ve now got generators and installers, thanks to our latest project [Igniter](https://hexdocs.pm/igniter)! First, we’ll cover why *I avoided them for so long*. Then we dive into how Igniter changes the game!

Igniter - Rethinking code generation with project patching

Human history is a story of us inventing new technologies, then learning how to get the most out of them. This merry dance has driven our civilisation.First a new technology comes along, disrupting old ways of working. As adoption spreads, it becomes refined and standardised. In turn, this opens the door for new possibilities. Sometimes these are small incremental improvements. Sometimes they have been breakthroughs that shaped the course of human history.

This is what happened in the industrial revolution. Steam engines, railways, the telegraph, iron and steel production, textile manufacturing machinery. All were major breakthroughs which paved the way for further advancements. Advancements which are often combined together in unpredictable ways.

<h3>Learning from other domains - construction</h3>

These kinds of technological step changes are happening in software right now - faster then we’ve experienced before. This rapid progress can be hard to understand when we&#39;re caught up in the middle of it. Historical lessons from other industries can help provide perspective on this.

Many comparisons are made between software and the building construction industry. Much of how software is built was originally derived from the construction industry. For both good and bad.

The most obvious are the lessons we’ve learnt about how not to manage software delivery, i.e. &#39;waterfall&#39;. The &#39;Agile&#39; movement is an example of software engineering coming of age. It&#39;s human responding to a new technology with new, improved ways of working.

Other comparisons give perspective about where software is at, and what might come next.

<h3>How we got to now - cities of glass</h3>

A key innovation in modern building construction was concrete combined with steel. Structural steel frames combined with reinforced concrete. This was the technological step change that led to the age of skyscrapers.

Previous to skyscrapers, buildings were all about bricks and mortar. Brick buildings are essentially masonry boxes. The walls themselves are structural and load bearing. They provide the support for the roof and everything else.

This creates two big constraints. Windows can only be so big before the wall loses its strength. Brick walls can only be so tall before they become unstable, limiting the building height. The result is that brick buildings are mostly square boxes, up to 4-5 stories tall, with rows of evenly spaced windows.

The introduction of steel and concrete changed how buildings are constructed. Now tall buildings are supported by a central, structural column. These columns are made out of steel frames and reinforced concrete, stretching hundreds of metres into the sky.

These structural columns are what allows us to step past the limited properties of brick walls. The external facade is no longer load bearing, holding up the weight of the building. This has led to an era where buildings are no longer short square, stone boxes. Now we have the glass covered towers of all shapes and sizes, that soar into the sky.

<h3>Where we to next</h3>

A recent innovation in the building industry is a <a href="https://builtoffsite.com.au/news/modular-construction-top-down-skyscraper/">technique for top-down construction</a>. It creates some crazy visuals of inverted buildings. 

First they build the structural spine of the building. Then each floor is assembled on the ground out of prefabricated components. When completed, the entire floor is lifted into place with jacks. They start building at the top floor and then work down to the ground. Working on the ground has all kinds of benefits, from speed, cost reduction and worker safety.

<img src="https://www.datocms-assets.com/55475/1719976856-untitled.png" alt="">
<img src="https://www.datocms-assets.com/55475/1719976844-untitled-1.png" alt="">
Source: <a href="https://builtoffsite.com.au/">buildoffsite.com.au</a>

To get to this point, a series of enabling technologies and <a href="https://alembic.com.au/blog/from-iron-to-algorithms-development-frameworks-are-transforming-software">engineering practices</a> were required. Reinforced concrete, interchangeable parts, modularisation, prefabrication, building standards and codes, etc. They all played a part in making this possible.

This construction approach breaks our mental model of how buildings are constructed. Literally flipping it upside down. It sets the brain racing about how this is possible. And what else might be possible in the future.

<h2>Application frameworks provide a leap forward for software</h2>

Just as skyscrapers now shape our broader skylines, technological tooling advancements are reshaping what modern applications look like. Software development is still young and evolving. We&#39;re constantly discovering new technologies and construction methods. These are changing the way we build software, extending what is possible.

We’re starting to find a way past the bricks and mortar boxes. We’re breaking down monolith applications, which are slow to construct and impossible to adapt over time. We’re discovering our equivalent of a building’s ”structural spine”.

Software development frameworks such as <a href="https://ash-hq.org/">Ash</a> are one of the ways we’re doing it. One of Ash&#39;s core innovations is that it provides that central structural spine. This supports the rapid compilation of modular components into a web app. It does this faster and with more flexibility than traditional coding methods.

It&#39;s the software equivalent of building a structural column that entire floors can be plugged into. Except, we&#39;re not talking about physical buildings where floors are locked in place. We&#39;re working in software. Ash&#39;s composability means extensions can be easily removed and replaced with another one at any time. Rather than being locked into a monolith, applications can evolve and change shape over time.

This is different to what has come before. This presents new possibilities for the way we build software applications. Not only is it providing immediate benefits, it&#39;s providing a platform for the future. It&#39;s stepping us forward into the next wave of innovations that allow software applications to soar up into the sky.

Human history is a story of us inventing new technologies, then learning how to get the most out of them. These technological step changes are happening in software right now - faster then we’ve experienced before. This rapid progress can be hard to understand when we're caught up in the middle of it. Historical comparisons from other industries can help provide perspective on where software is at - and what might come next.

Do your tech choices empower a test and learn cycle?

Want to read more?