twitter:title

Articles about functional programming including Elixir, TypeScript, GraphQL, React and more.

description

twitter:description

https://www.datocms-assets.com/55475/1631844864-alembic-logo-svg.svg?auto=format&fit=max&w=1200

twitter:image

twitter:image:alt

twitter:card

twitter:site

We value learning and experimentation to make sure we always provide the best technology solutions for our partners. We highlight some of our favorite experiments on our engineering blog.

Engineering Blog

<p>Authentication is a crucial part of most web applications. With the <a href="https://alembic.com.au/blog/announcing-ash-authentication">release of Ash Authentication</a>, we are now able to authenticate users in Elixir applications using simple DSL on Ash resources. Ash Authentication comes with default configurations that work well for most applications. However, in some cases, customisation is necessary to cater to specific business needs. </p>

<p>In this blog post, we will discuss how to customise Ash authentication with Phoenix LiveView which follows the <a href="https://realworld-docs.netlify.app/docs/specs/backend-specs/endpoints#registration">RealWorld specs</a>. This required some deviation from the Ash Authentication defaults.</p>

<p>Specifically, how do we go from this:</p>

<p><img src="https://raw.githubusercontent.com/twillzy/custom-auth-ash/main/assets/from.png" alt="Default UI for Ash Authentication"></p>

<p>to this:</p>

<p><img src="https://raw.githubusercontent.com/twillzy/custom-auth-ash/main/assets/to.png" alt="Custom UI for Ash Authentication"></p>

<p>Before we dive into customisation, let&#39;s first have a brief overview of Ash Authentication. Ash Authentication provides features such as registration and sign-in using passwords/OAuth 2.0. Ash Authentication also offers default LiveView components for login, logout, registration, and password reset. These default configurations work well for most applications, but in some cases, they need to be customised.</p>

<h2>Prerequisite</h2>

<p>A working Phoenix + Ash application with default Ash Authentication. <em>You can achieve this by following this <a href="https://ash-hq.org/docs/guides/ash_authentication_phoenix/latest/tutorials/getting-started-with-ash-authentication-phoenix">step-by-step tutorial</a>.</em> Otherwise, you can also skip the tutorial and checkout the code on this <a href="https://github.com/twillzy/custom-auth-ash/tree/default-ash-auth">branch</a> on GitHub.</p>

<h2>Customising Ash Authentication</h2>

<p>First, let&#39;s modify the registration process. In the default process, a user enters their email address, password and password confirmation. However, in our case, we will skip the password confirmation and ask for a username instead. To achieve this, we can modify the User resource at <code>custom_auth/lib/custom_auth/accounts/resources</code>.</p>

<p>Let&#39;s add a new User attribute, username, in the User resource.</p>

<pre><code class="elixir">attributes do
  uuid_primary_key(:id)
  attribute(:email, :ci_string, allow_nil?: false)
  attribute(:hashed_password, :string, allow_nil?: false, sensitive?: true)
  attribute(:username, :string, allow_nil?: false)
end
</code></pre>

<p>Generate and run a new migration:</p>

<pre><code class="sh">mix ash_postgres.generate_migrations --name add_username_to_users
mix ash_postgres.migrate
</code></pre>

<p>At this point, you should be able to register a new user with username, email and password. Let&#39;s confirm that. Fire up an IEx shell by running:</p>

<pre><code class="sh">iex -S mix phx.server
</code></pre>

<p>Then,</p>

<pre><code class="elixir">CustomAuth.Accounts.User |&gt; Ash.Changeset.for_create(:register_with_password, %{email: &quot;abc@gmail.com&quot;, password: &quot;password&quot;, username: &quot;username123&quot;}) |&gt; CustomAuth.Accounts.create!()
</code></pre>

<p>Oops, Ash is telling us that <code>Invalid value provided for password_confirmation: does not match</code>. We need to skip <code>password_confirmation</code> and we can do that by setting <code>confirmation_required?</code> to <code>false</code> like so:</p>

<pre><code class="elixir">strategies do
  password :password do
    identity_field(:email)
    confirmation_required?(false)
    register_action_accept([:username])
  end
end
</code></pre>

<p>Now, you can confirm that the password_confirmation field is no longer required to register a new user:</p>

<pre><code class="elixir">CustomAuth.Accounts.User
|&gt; Ash.Changeset.for_create(:register_with_password, %{email: &quot;abc@gmail.com&quot;, password: &quot;password&quot;, username: &quot;username123&quot;})
|&gt; CustomAuth.Accounts.create!()
</code></pre>

<p>🥳🥳🥳 Well done! That is all you need to modify your Ash user resource to customise the registration process. We will move on to changing the form on our frontend to support this new process. To begin, let&#39;s get the styling out of the way and include a stylesheet in <code>lib/custom_auth_web/components/layouts/root.html.heex</code>. </p>

<pre><code>&lt;link rel=&quot;stylesheet&quot; href=&quot;//demo.productionready.io/main.css&quot; /&gt;
</code></pre>

<p>Next, We need to override the default behaviour of the register process given to us by Ash Authentication Phoenix. Let&#39;s say goodbye to our helpful friend <code>sign_in_route()</code> and replace it with 2 new routes <code>register</code> and <code>sign-in</code> at <code>custom_auth/lib/custom_auth_web/router.ex</code></p>

<pre><code class="elixir">scope &quot;/&quot;, CustomAuthWeb do
    ....

    # sign_in_route() &lt;--- remove this line
    live &quot;/register&quot;, AuthLive.Index, :register
    live &quot;/sign-in&quot;, AuthLive.Index, :sign_in

    ....
  end
</code></pre>

<p>The last two steps are simply copying and pasting some html code. As this is not a Phoenix LiveView tutorial, I will only highlight the important changes related to Ash Authentication when we are done with the copy paste. For now, go ahead and copy the code below and paste it in a new file <code>lib/custom_auth_web/live/auth_live/index.ex</code>. </p>

<pre><code class="elixir">defmodule CustomAuthWeb.AuthLive.Index do
  use CustomAuthWeb, :live_view

  alias CustomAuth.Accounts
  alias CustomAuth.Accounts.User
  alias AshPhoenix.Form

  @impl true
  def mount(_, _, socket) do
    {:ok, socket}
  end

  @impl true
  def handle_params(params, _url, socket) do
    {:noreply, apply_action(socket, socket.assigns.live_action, params)}
  end

  defp apply_action(socket, :register, _params) do
    socket
    |&gt; assign(:form_id, &quot;sign-up-form&quot;)
    |&gt; assign(:cta, &quot;Sign up&quot;)
    |&gt; assign(:alternative_path, ~p&quot;/sign-in&quot;)
    |&gt; assign(:alternative, &quot;Have an account?&quot;)
    |&gt; assign(:action, ~p&quot;/auth/user/password/register&quot;)
    |&gt; assign(
      :form,
      Form.for_create(User, :register_with_password, api: Accounts, as: &quot;user&quot;)
    )
  end

  defp apply_action(socket, :sign_in, _params) do
    socket
    |&gt; assign(:form_id, &quot;sign-in-form&quot;)
    |&gt; assign(:cta, &quot;Sign in&quot;)
    |&gt; assign(:alternative_path, ~p&quot;/register&quot;)
    |&gt; assign(:alternative, &quot;Need an account?&quot;)
    |&gt; assign(:action, ~p&quot;/auth/user/password/sign_in&quot;)
    |&gt; assign(
      :form,
      Form.for_action(User, :sign_in_with_password, api: Accounts, as: &quot;user&quot;)
    )
  end

  @impl true
  def render(assigns) do
    ~H&quot;&quot;&quot;
    &lt;div class=&quot;auth-page&quot;&gt;
      &lt;div class=&quot;container page&quot;&gt;
        &lt;div class=&quot;row&quot;&gt;
          &lt;div class=&quot;col-md-6 offset-md-3 col-xs-12&quot;&gt;
            &lt;h1 class=&quot;text-xs-center&quot;&gt;&lt;%= @cta %&gt;&lt;/h1&gt;
            &lt;p class=&quot;text-xs-center&quot;&gt;
              &lt;a href={@alternative_path}&gt;&lt;%= @alternative %&gt;&lt;/a&gt;
            &lt;/p&gt;

            &lt;.live_component
              module={CustomAuthWeb.AuthLive.AuthForm}
              id={@form_id}
              form={@form}
              is_register?={@live_action == :register}
              action={@action}
              cta={@cta}
            /&gt;
          &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
    &quot;&quot;&quot;
  end
end
</code></pre>

<p>And the code below and paste it in a new file <code>lib/custom_auth_web/live/auth_live/auth_form.ex</code>. </p>

<pre><code class="elixir">defmodule CustomAuthWeb.AuthLive.AuthForm do
  use CustomAuthWeb, :live_component
  use Phoenix.HTML
  alias AshPhoenix.Form

  @impl true
  def update(assigns, socket) do
    socket =
      socket
      |&gt; assign(assigns)
      |&gt; assign(trigger_action: false)

    {:ok, socket}
  end

  @impl true
  def handle_event(&quot;validate&quot;, %{&quot;user&quot; =&gt; params}, socket) do
    form = socket.assigns.form |&gt; Form.validate(params, errors: false)

    {:noreply, assign(socket, form: form)}
  end

  @impl true
  def handle_event(&quot;submit&quot;, %{&quot;user&quot; =&gt; params}, socket) do
    form = socket.assigns.form |&gt; Form.validate(params)

    socket =
      socket
      |&gt; assign(:form, form)
      |&gt; assign(:errors, Form.errors(form))
      |&gt; assign(:trigger_action, form.valid?)

    {:noreply, socket}
  end

  @impl true
  def render(assigns) do
    ~H&quot;&quot;&quot;
    &lt;div&gt;
      &lt;ul class=&quot;error-messages&quot;&gt;
        &lt;%= if @form.errors do %&gt;
          &lt;%= for {k, v} &lt;- @errors do %&gt;
            &lt;li&gt;
              &lt;%= humanize(&quot;#{k} #{v}&quot;) %&gt;
            &lt;/li&gt;
          &lt;% end %&gt;
        &lt;% end %&gt;
      &lt;/ul&gt;
      &lt;.form
        :let={f}
        for={@form}
        phx-change=&quot;validate&quot;
        phx-submit=&quot;submit&quot;
        phx-trigger-action={@trigger_action}
        phx-target={@myself}
        action={@action}
        method=&quot;POST&quot;
      &gt;
        &lt;%= if @is_register? do %&gt;
          &lt;fieldset class=&quot;form-group&quot;&gt;
            &lt;%= text_input(f, :username,
              class: &quot;form-control form-control-lg&quot;,
              placeholder: &quot;Username&quot;
            ) %&gt;
          &lt;/fieldset&gt;
        &lt;% end %&gt;
        &lt;fieldset class=&quot;form-group&quot;&gt;
          &lt;%= text_input(f, :email,
            class: &quot;form-control form-control-lg&quot;,
            placeholder: &quot;Email&quot;
          ) %&gt;
        &lt;/fieldset&gt;
        &lt;fieldset class=&quot;form-group&quot;&gt;
          &lt;%= password_input(f, :password,
            class: &quot;form-control form-control-lg&quot;,
            placeholder: &quot;Password&quot;
          ) %&gt;
        &lt;/fieldset&gt;
        &lt;%= submit(@cta, class: &quot;btn btn-lg btn-primary pull-xs-right&quot;) %&gt;
      &lt;/.form&gt;
    &lt;/div&gt;
    &quot;&quot;&quot;
  end
end
</code></pre>

<p>And here is the most important bit on the frontend: to utilise <code>AshPhoenix.Form</code> to deal with Ash changesets.</p>

<pre><code class="elixir">|&gt; assign(:action, ~p&quot;/auth/user/password/register&quot;)
|&gt; assign(
  :form,
  Form.for_create(User, :register_with_password, api: Accounts, as: &quot;user&quot;)
)
</code></pre>

<p>That is it! At this point, we should be able to register a new account with email, username and password and sign in to it as well.</p>

<h2>Conclusion</h2>

<p>Customising Ash Authentication using the Ash framework and Phoenix LiveView is a straightforward process. By modifying the default templates and controllers, we can cater to specific business needs and add additional layers of security. Ash Authentication provides a solid foundation for web application security, and with the power of Phoenix LiveView, we can create reactive, real-time web applications that are both secure and user-friendly.</p>


Authentication is a crucial part of most web applications. With the release of Ash Authentication, we are now able to authenticate users in Elixir applications using simple DSL on Ash resources. In this blog post, we will show you how to customise Ash Authentication with Phoenix LiveView.


Customising Ash Authentication with Phoenix LiveView

<p>We&#39;re extremely pleased to announce the first public release of <a href="https://github.com/team-alembic/ash_authentication/"><code>ash_authentication</code></a> and <a href="https://github.com/team-alembic/ash_authentication_phoenix/"><code>ash_authentication_phoenix</code></a>. Ash Authentication allows you to authenticate users in your Elixir applications using a simple DSL on your Ash resources.</p>

<p>Using Ash Authentication saves you the time, effort and maintenance burden of implementing your own solution. This improves significantly upon hand rolling or generating code because you rely on a library to implement authentication rather than managing that code yourself making it much more secure.</p>

<h2>What is Ash?</h2>

<p><a href="https://ash-hq.com">Ash Framework</a> is a declarative, resource-oriented application development framework for <a href="https://elixir-lang.org">Elixir</a>. Resources can model anything; a database table, an external API, or even your own code.  See the <a href="https://elixirforum.com/t/ash-framework-a-declarative-resource-oriented-application-development-framework-for-elixir/51119">recent 2.0 announcement here</a>.</p>

<p>You can think of Ash as a standardised way to model and build your applications by composing features together. It takes care of the common requirements of building an Elixir application so you can focus on building the parts that make your application special.</p>

<p>Ash Authentication fits neatly into the Ash ecosystem by defining a handful of extensions which can modify Ash resources to add new functionality. The ability of Ash to be extended is what makes Ash so powerful. In fact, Ash’s own resources and APIs are extensions!</p>

<p><img src="https://www.datocms-assets.com/55475/1673928920-ash-auth-logo.svg" alt="Ash Authentication padlock logo"></p>

<h2>What is Ash Authentication?</h2>

<p>Ash Authentication is a drop-in authentication solution for users of the Ash framework who want to provide <strong>password-based</strong> or <strong>social sign-in via OAuth 2.0</strong>.</p>

<pre><code class="elixir">defmodule MyApp.Accounts.User do
  use Ash.Resource, extensions: [AshAuthentication]

  authentication do
        api MyApp.Accounts

    strategies do
      password do
        identity_field :email
      end
    end
  end
end
</code></pre>

<p>Current features include:</p>

<ul>
<li>Registration and sign-in using passwords.</li>
<li>Registration and sign-in using OAuth 2.0.</li>
<li>Confirmation and reset flows.</li>
<li>Extensible LiveView components.</li>
<li>Predefined configuration for Auth0 and GitHub sign-in with more coming soon.</li>
<li>Works with or without Phoenix.</li>
</ul>

<h2>Why Ash Authentication?</h2>

<p>Critically, Ash Authentication is not a code generator, but an extension to the Ash ecosystem of declaratively defined applications. This means that you’ll receive security updates, performance improvements or new features just like any other Elixir library - with a simple <code>mix deps.update</code>.  Additionally, it can be extended to provide new and novel authentication systems by third-party extensions or your own team.</p>

<p>Importantly, all the authentication code lives in the library, so you’ll never have to manage generated or hand written code in your own app. Stop worrying about security or updating generated code as it drifts over time.</p>

<h2>Future plans</h2>

<p>We have a lot of exciting new features planned for Ash Authentication, including two factor authentication, lots more pre-configured social sign in options, and the ability to act as an OAuth2 provider allowing other apps to sign into your app.</p>

<p>If you feel Ash Authentication is missing something important then please feel free to <a href="https://github.com/team-alembic/ash_authentication/issues">raise an issue</a>.</p>

<h2>Can I use it in my app?</h2>

<p>Yes. We’ve put a lot of effort into making sure that the code is well structured and safe by default. It is in production already in several applications, including <a href="https://ash-hq.org/">Ash HQ</a> and a handful of community owned apps from which we&#39;ve received and addressed a lot of feedback. With that said, this is a new library that hasn’t seen wider production usage, so by all means try it out but please don&#39;t use it for anything you would consider mission critical just yet.</p>

<p>With the help of the community, we’ll be addressing feedback and issues as they arise. The benefit of the Ash approach is that any issues that are resolved and released will be available to your app via a simple upgrade of the Ash Auth dependency.</p>

<h2>I’m interested, now what?</h2>

<p>The first place to start would be our <a href="https://ash-hq.org/docs/guides/ash_authentication/latest/tutorials/getting-started-with-authentication">Getting Started Guide</a>, followed by the <a href="https://ash-hq.org/docs/module/ash_authentication/latest/ashauthentication">extension documentation</a>.  If you run into any trouble - or just want a chat - we’d love to help over on the <a href="https://discord.gg/NnkhUmkv">Ash Discord</a>.</p>

<p>😎 Happy Coding! 😎</p>

<h2>Links</h2>

<ul>
<li><a href="https://hex.pm/packages/ash_authentication">https://hex.pm/packages/ash_authentication</a></li>
<li><a href="https://hex.pm/packages/ash_authentication_phoenix">https://hex.pm/packages/ash_authentication_phoenix</a></li>
<li><a href="https://github.com/team-alembic/ash_authentication">https://github.com/team-alembic/ash_authentication</a></li>
<li><a href="https://github.com/team-alembic/ash_authentication_phoenix">https://github.com/team-alembic/ash_authentication_phoenix</a></li>
<li><a href="https://ash-hq.org/docs/guides/ash_authentication/latest/tutorials/getting-started-with-authentication">Getting Started Guide</a></li>
<li><a href="https://ash-hq.org/docs/module/ash_authentication/latest/ashauthentication">Ash Auth Documentation</a></li>
<li><a href="https://ash-hq.org">https://ash-hq.org</a></li>
<li><a href="https://alembic.com.au">https://alembic.com.au</a></li>
<li><a href="https://discord.gg/NnkhUmkv">Ash Discord</a></li>
</ul>

<h2>Alembic</h2>

<p>Alembic is a globally distributed software consultancy. We specialise in Elixir, Phoenix and Ash and have sponsored the development of Ash Authentication as part of our ongoing commitment to the Ash ecosystem.</p>

<p>If you need help with your Elixir, Phoenix or Ash application development then please <a href="https://alembic.com.au/contact">reach out to us</a>.</p>


We're extremely pleased to announce the first public release of `ash_authentication` and `ash_authentication_phoenix`. Ash Authentication allows you to authenticate users in your Elixir applications using a simple DSL on your Ash resources.

Announcing Ash Authentication

<p>This is Part two of our blog post comparing Elixir and Python when applied to a real world AI/ML problem. If you haven’t read it check out the <a href="https://alembic.com.au/blog/elixir-vs-python-for-real-world-ai-ml-part-1">first part</a>.</p>

<p>Just to recap, we&#39;re trying to train a model to predict the water level of a dam. In part one we fetched the data from an API, cleaned it and combined it into a single DataFrame. In part two we’re going to build, train and evaluate a neural network using both Python and Elixir. Then we’ll take a look back at the Elixir and Python implementations and compare and contrast the two ecosystems.</p>

<h2>Prepare data for the model</h2>

<p>We have our data neatly in one DataFrame (<code>df</code> in the code below). Now we need to sort it into:</p>

<ul>
<li>Training data</li>
<li>Testing data</li>
<li>Validation data (just in the Elixir Implementation)</li>
</ul>

<p>Each of these categories are subdivided further into:</p>

<ul>
<li>Features (input to the model)</li>
<li>Labels (expected output of the model)</li>
</ul>

<p><strong>We want 80% of our data for training and 20% for testing.</strong> The training data will be split further so that <strong>20% of the training data is used for validation</strong> while training. The features will be the rainfall and stream level data. The labels will be the water level of the dam.</p>

<p>Here we’ll be using the Elixir library <a href="https://github.com/elixir-nx/nx">Nx</a>. <code>Nx</code> is a multi-dimensional tensor library for Elixir. The real beauty of it is that the tensors can target three backends, Elixir Native code, <a href="https://www.tensorflow.org/xla">Google XLA</a> (the same backend as TensorFlow) and <a href="https://pytorch.org/cppdocs/installing.html">LibTorch</a>. Enabling your neural networks to have similar performance to <a href="https://pytorch.org/">PyTorch</a> and <a href="https://www.tensorflow.org/">TensorFlow</a>. <a href="https://alembic.com.au/blog/high-performance-numerical-elixir-with-nx">See our introductory post about Nx.</a></p>

<p>Here’s the Python code:</p>

<pre><code class="python">feature_columns = [
  &quot;v_568051_10&quot;, &quot;v_568045_10&quot;, &quot;v_563079_10&quot;,
  &quot;v_563046_10&quot;, &quot;v_563035_10&quot;, &quot;v_212250_100&quot;,
  &quot;v_212270_100&quot;]

label_column = &quot;water_level_difference&quot;
model_columns = feature_columns + [label_column]

train_data = df.copy()[model_columns].sample(frac=0.8, random_state=12345)
test_data = df.copy()[model_columns].drop(train_data.index)

train_features = train_data.copy()
test_features = test_data.copy()

train_labels = train_features.pop(&quot;water_level_difference&quot;).values.reshape(-1, 1)
test_labels = test_features.pop(&quot;water_level_difference&quot;).values.reshape(-1, 1)
</code></pre>

<p>And the Elixir code:</p>

<pre><code class="elixir"># snippet of the helper module
defmodule Helper do

    ...

    def split_data(df, decimal) do
        row_no = DF.n_rows(df)

    {first, second} =
      df
      |&gt; DF.to_rows()
      |&gt; Enum.shuffle()
      |&gt; Enum.split(round(decimal * row_no))

    {DF.new(first), DF.new(second)}
  end

  def df_to_batches(df, feature_cols, label_col, batch_size \\ 1) do
    Stream.zip(
      df_to_tensor_batches(df[feature_cols], batch_size),
      df_to_tensor_batches(df[[label_col]], batch_size)
    )
  end

  def df_to_tensor_batches(df, batch_size) do
    df
    |&gt; df_to_tensor()
    |&gt; Nx.shuffle(axis: 0)
    |&gt; Nx.to_batched(batch_size, leftover: :discard)
  end

    ...

end

feature_columns = [
  &quot;v_568051_10&quot;,
  &quot;v_568045_10&quot;,
  &quot;v_563079_10&quot;,
  &quot;v_563046_10&quot;,
  &quot;v_563035_10&quot;,
  &quot;v_212250_100&quot;,
  &quot;v_212270_100&quot;
]

label_column = &quot;water_level_difference&quot;
model_columns = [label_column | feature_columns]

df = DF.select(df, model_columns)

{train_df, test_df} = Helper.split_data(df, 0.8)
{train_df, validation_df} = Helper.split_data(train_df, 0.8)

training_batches = Helper.df_to_batches(train_df, feature_columns, label_column, 8)
validation_batches = Helper.df_to_batches(validation_df, feature_columns, label_column, 1)
testing_batches = Helper.df_to_batches(test_df, feature_columns, label_column, 1)
</code></pre>

<p>As you can see the Elixir code is a little more involved. <code>Axon</code> (Elixir’s neural network library) is choosey about the format it receives the data in, it won’t take an <code>Explorer</code> DataFrame, only zipped batches of  <code>Nx</code> tensors. However, there isn’t an obvious and optimal way to convert an <code>Explorer</code> data frame into an <code>Nx</code> tensor. The same is true for splitting the data - there’s no out-of-the-box way of doing it.</p>

<h2>Building the model</h2>

<p>Now we have all our data in the right format we are ready to build our neural network. Funnily enough this is actually the easiest part. In our Python version we will use <code>TensorFlow</code> and in Elixir we’re going to use <code>Axon</code>.</p>

<p><a href="https://github.com/elixir-nx/axon"><code>Axon</code></a> is a neural network library built completely on top of <code>Nx</code>. <code>Axon</code> has a bunch of sensible APIs that are simple enough for a beginner but flexible enough that an expert can easily do <em>just</em> what they want.</p>

<p>Let’s take a look at the TensorFlow model in Python first, then compare with Elixir.</p>

<p>Python:</p>

<pre><code class="python">normaliser = tf.keras.layers.Normalization(axis=1)
normaliser.adapt(train_features)

test_model = tf.keras.Sequential(name=&quot;stream_and_rain_model&quot;, layers=[
    layers.Input(shape=(7,)),
    normaliser,
    layers.Dropout(rate=0.5),
    layers.Dense(units=16, activation=&quot;relu&quot;),
    layers.Dropout(rate=0.5),
    layers.Dense(units=1)
])
</code></pre>

<p>Elixir:</p>

<pre><code class="elixir">model =
  Axon.input(&quot;stream_and_rain_model&quot;)
  |&gt; Axon.dropout(rate: 0.5)
  |&gt; Axon.dense(16)
    |&gt; Axon.relu()
  |&gt; Axon.dropout(rate: 0.5)
  |&gt; Axon.dense(1)
</code></pre>

<p>You can see how these are both remarkably similar, the only major difference is that the Python model has a normalising layer. There isn’t a layer for this in <code>Axon</code> yet, as it’s simple enough to do the transformation yourself before the features are input to the neural network - the layer would be a nice quality of life improvement though.</p>

<h2>Training</h2>

<p>Python:</p>

<pre><code class="python">test_model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss=&#39;mean_absolute_error&#39;)

history = test_model.fit(
    train_features.values,
    train_labels,
    epochs=30,
    # Suppress logging.
    verbose=1,
    # Calculate validation results on 20% of the training data.
    validation_split = 0.2)
</code></pre>

<p>Elixir:</p>

<pre><code class="elixir">model_params =
  model
  |&gt; Axon.Loop.trainer(:mean_absolute_error, Axon.Optimizers.adam(0.001))
  |&gt; Axon.Loop.validate(model, validation_batches)
  |&gt; Axon.Loop.metric(:mean_absolute_error, &quot;validation_loss&quot;)
  |&gt; Axon.Loop.run(training_batches, %{}, epochs: 30)
</code></pre>

<p>Again, the semantics are almost exactly the same. Python does a little more for you here - remember how we didn’t need to make a <code>validation_batches</code> variable? That’s because we can simply tell Python to validate while training for us.</p>

<h2>Results</h2>

<p><strong>Both the models are training well, we can see the reduction in loss and validation loss</strong> over the epochs. While you do get this information printed in the terminal in <code>Axon</code>, there wasn’t a clear way to extract it, so the chart below is from the Python code.</p>

<p>Here is the Python code:</p>

<pre><code class="python">hist_df = pd.DataFrame(history.history)
hist_df[&quot;epoch&quot;] = history.epoch
hist_df.rename(
    columns={&quot;loss&quot;:&quot;training_loss&quot;, &quot;val_loss&quot;:&quot;validation_loss&quot;},
    inplace=True
)

alt.Chart(hist_df).mark_line().transform_fold(
    fold=[&#39;training_loss&#39;, &#39;validation_loss&#39;], 
    as_=[&#39;variable&#39;, &#39;loss&#39;]
).encode(
    x=&quot;epoch:Q&quot;,
    y=&quot;loss:Q&quot;,
    color=&quot;variable:N&quot;
)
</code></pre>

<p>It produces:</p>

<p><img src="https://www.datocms-assets.com/55475/1677730463-training-results.svg" alt="Training results"></p>

<p>We can clearly see the training loss and the validation loss going down.</p>

<p><strong>We can use our test data to evaluate our model.</strong></p>

<p>Python:</p>

<pre><code class="python">test_model.evaluate(test_features, test_labels)
# 29/29 [==============================] - 0s 1ms/step - loss: 0.0208
</code></pre>

<p>Elixir:</p>

<pre><code class="elixir">model
|&gt; Axon.Loop.evaluator()
|&gt; Axon.Loop.metric(:mean_absolute_error)
|&gt; Axon.Loop.run(testing_batches, model_params, epoch: 1)
# Batch: 927, mean_absolute_error: 0.0245324
</code></pre>

<p><strong>These loss values of 0.0208 and 0.0245 mean that on average the model is within approximately 0.02m of the real change in dam water level,</strong> not too bad.</p>

<p><strong>We can observe this visually by feeding the model all of the feature data</strong>, and then comparing the predictions of the change in water level with the real change in water level.</p>

<p>In Python we are using <a href="https://altair-viz.github.io/">Altair</a> (a <a href="https://vega.github.io/vega-lite/">Vega-Lite</a> binding). In Elixir we’re using <a href="https://github.com/livebook-dev/vega_lite">VegaLite</a> (an Elixir wrapper around Vega-Lite), so it produces near identical graphs.</p>

<p>Here is the Python code for that:</p>

<pre><code class="python">y = test_model.predict(df[feature_columns])

compare_df = pd.DataFrame({
    &quot;t&quot;: df[[&quot;t&quot;]].values.flatten(),
    &quot;actual&quot;: df[[&quot;water_level_difference&quot;]].values.flatten(),
    &quot;prediction&quot;: y.flatten()
})

base = alt.Chart(compare_df.reset_index()[0:300]).encode(
    x=&quot;index:Q&quot;
)

(base.mark_line().encode(
    y=&quot;actual:Q&quot;
) + base.mark_line(color=&quot;orange&quot;).encode(
    y=&quot;prediction:Q&quot;
)).interactive()
</code></pre>

<p>And the Elixir code:</p>

<pre><code class="elixir">defmodule Helper do
    ...

  def df_to_tensor(df) do
    df
    |&gt; DF.names()
    |&gt; Enum.map(&amp;Explorer.Series.to_tensor(df[&amp;1]))
    |&gt; Nx.stack(axis: 1)
  end

    ...
end

all_features = df_to_tensor(df[feature_columns])

all_labels =
  df[label_column]
  |&gt; Series.to_tensor()
  |&gt; Nx.to_flat_list()

{_init_fn, predict_fn} = Axon.build(model, mode: :inference)

predictions =
  predict_fn.(model_params, all_features)
  |&gt; Nx.to_flat_list()

row_no = 4640

chart_data =
  DataFrame.new(
    prediction: predictions,
    actual: all_labels,
    count: Enum.map(1..row_no, &amp; &amp;1)
  )
  |&gt; DataFrame.to_rows()

Vl.new(width: 400, height: 400)
|&gt; Vl.data_from_values(Enum.take(chart_data, 300))
|&gt; Vl.layers([
  Vl.new()
  |&gt; Vl.param(&quot;prediction_chart&quot;, select: :interval, bind: :scales, encodings: [&quot;x&quot;, &quot;y&quot;])
  |&gt; Vl.encode(:y, field: &quot;prediction&quot;, type: :quantitative)
  |&gt; Vl.encode(:x, field: &quot;count&quot;, type: :quantitative)
  |&gt; Vl.mark(:line, color: &quot;orange&quot;),
  Vl.new()
  |&gt; Vl.encode(:x, field: &quot;count&quot;, type: :quantitative)
  |&gt; Vl.encode(:y, field: &quot;actual&quot;, type: :quantitative)
  |&gt; Vl.mark(:line, color: &quot;blue&quot;)
])
</code></pre>

<p>This produces:</p>

<p><img src="https://www.datocms-assets.com/55475/1677730467-actual-v-prediction.svg" alt="Actual water level vs predicted"></p>

<p><strong>The orange line is the model’s prediction, the blue line is the ground truth.</strong> <strong>We can see that it seems to predict large spikes quite well</strong>, but not necessarily their magnitude. The model seems to have a slight systematic bias towards predicting lower values than reality. I think this may be because our dataset is incomplete as we don’t have data about the outflow of the dam.</p>

<p><strong>I think for a model where the input data is only from the day before and its predicting the next day, it’s performing remarkably well.</strong> This model could be improved, of course, with more data sources but also by inputting multiple days worth of data at once. The purpose of this post was to compare Python and Elixir with a real problem, so that’s outside the scope of this post. If I’ve piqued your interest you can see the <a href="https://github.com/team-alembic/warragamba-blogpost">repo</a> (including a multi-day model in Python so the accuracy is better).</p>

<h2>Elixir is punching way above its weight</h2>

<p><strong>I was not expecting Elixir to be better than Python for data science</strong>, and right now it isn’t. The Python data science ecosystem is huge and has had a vast amount of money poured into it by big tech. However <strong>it’s important to acknowledge just how far Elixir’s data science ecosystem has come</strong> in the past two years. <strong>It’s gone from zero to being a totally viable option for a data science project.</strong> It’s done this by standing on the shoulders of giants, taking all the best bits from the whole ecosystem - packaging them together in a simple but flexible way.</p>

<h2>The Elixir data science ecosystem has early adopter challenges</h2>

<p>Being an early adopter in a young ecosystem will always be harder than participating in a mainstream one. The Elixir data science ecosystem is no different. While you’ll find lots of excitement online, you might not be able to find a blog or forum post with the information you need to do ‘X’, or how to get rid of error ‘Y’. <strong>Fortunately a lot of love has been given to the documentation on these projects - practically everything is documented</strong>. The <a href="https://hexdocs.pm/explorer/exploring_explorer.html">quick</a> <a href="https://hexdocs.pm/nx/intro-to-nx.html#content">start</a> <a href="https://hexdocs.pm/axon/your_first_axon_model.html">guides</a> and working examples are especially helpful.</p>

<p>Another niggle I’ve found is that there is some ‘glue’ missing between the libraries within the ecosystem. For example, why is there no function to turn an <code>Explorer</code> data frame into a 2D <code>Nx</code> tensor? It seems like an obvious omission. And why doesn’t some kind of <code>test_train_split/2</code> function exist within <code>Axon</code>? These are just a couple of example friction points and there may be good reasons for them, but I did find myself having to build my own, probably inefficient solutions, for something that should already exist.</p>

<p><strong>Its not a deal breaker, its the side effect of a young and exciting ecosystem</strong>, but the lack of small developer conveniences does break your flow when working on data engineering problems. As the community and ecosystem mature we should see the developer experience improve.</p>

<h2>It feels better to program in Elixir</h2>

<p>I may well be biased, but it felt way better programming in Elixir. I found the code easier to reason about and more understandable.</p>

<p><strong>The main reason for this is that the Elixir was far more readable</strong>, just look at this example:</p>

<pre><code class="elixir"># python
water_level_df = water_level_df[~water_level_df[&quot;q&quot;].isin([201,205])]

# elixir
water_level_df = DF.filter(water_level_df, q != 201 and q != 255)
</code></pre>

<p>Both of these are one liners. But <strong>the Elixir is much more self-explanatory</strong>, even a junior developer would know exactly what the code was doing without prior knowledge.</p>

<p>I also found that I was never looking over my shoulder, as I was in Python, considering if any of the functions returned a reference to an object, or whether the data was copied then modified. I found this especially with <a href="https://pandas.pydata.org/">pandas</a> and it actually caused a couple of pesky bugs, these bugs encourage you to defensively <code>.copy()</code> your data which isn’t good for performance. In contrast, <strong>in Elixir I knew that I was always passing an immutable object. It removed a whole category of errors from the project.</strong></p>

<h2>Conclusion</h2>

<p>All in all this was a rewarding experience. I’d still say for now, most of the time, Python is a safe option. From the outset you know it can do whatever you need it to. But <strong>if you know it can be done with the current Elixir tooling, I’d go for it</strong> - Elixir is much more enjoyable to work in. Where I think the Elixir data science tools can really shine is when you already have an Elixir codebase (e.g. a web app) and you want to incorporate some data driven feature (e.g. recommendations). This is now viable to do in-app without having the burden of spinning off a Python microservice to process your data.</p>

<p>The Elixir data science tooling and community is still very young. <strong>Two years ago it was practically non-existent and now it’s a viable and performant option.</strong> <strong>Python has had a ten year head start but if it carries on at this rate I may be telling you Elixir is the clear winner before too long.</strong></p>


In the last part of this series, we will explore a real-world machine learning problem in Python and Elixir. By the end you will have two end-to-end examples of how you can combine clean and unify data sources. Then use that data to train and evaluate a neural network.

Elixir vs Python for real world AI/ML (Part 2)

<p><a href="https://www.phoenixframework.org">Phoenix LiveView</a> is a web application framework that lets us produce server-rendered HTML and just a little JavaScript to create applications that feel responsive, provide fast feedback to users and can react to changes occurring on the server.</p>

<p>In this post, we’ll touch on LiveNavigation, latency and how to mask it with page transitions during loading. We’ll also have some fun with transitions for adding and removing table rows.</p>

<p>The code shown in this post is based on the output of the Phoenix generators <code>mix phx.new</code> and <code>mix phx.gen.live</code> prior to the release of v1.7.0 built from commit (6811f0ea). By the time Phoenix 1.7 is officially released, there may be some changes to the output of the generators.</p>

<h2>Live Navigation</h2>

<p>LiveView provides multiple options for navigating between pages, depending on whether it is being initiated from the client or server, and how much of the page needs to be re-loaded:</p>

<ul>
<li>A full page load can be initiated from client side with a <code>&lt;.link href={...}&gt;</code> component, or from the server with the <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#redirect/2">redirect/2</a> function.</li>
<li>A new LiveView can be mounted within the existing page layout from the client with <code>&lt;.link navigate={...}&gt;</code> or <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.JS.html#navigate/1">JS.navigate/1</a> or from the server with the <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#push_navigate/2">push_navigate/2</a> function.</li>
<li>The existing LiveView can be updated without re-mounting from the client with <code>&lt;.link patch={...}/&gt;</code> or <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.JS.html#patch/1">JS.patch/1</a> or from the server with the <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#push_patch/2">push_patch/2</a> function.</li>
</ul>

<p>See the <a href="https://hexdocs.pm/phoenix_live_view/live-navigation.html">LiveNavigation guide</a> or <a href="https://medium.com/pragmatic-programmers/understanding-liveview-navigation-2b004a0e100b">Understanding LiveView Navigation</a> for more information.</p>

<h2>Latency</h2>

<p>Applications that primarily use the <code>&lt;.link navigate={...}/&gt;</code> method for navigation have a fairly smooth user experience, particularly when latency is low and the client can update the UI almost immediately. However, the user experience degrades as the distance between the user and the LiveView server increases, causing noticeable latency.</p>

<p>In a demo application deployed to the us-west-2 AWS region, the latency for navigating between LiveViews experienced by a user in east coast Australia was measured at ~500ms.
With a 500ms latency, the page no longer feels like it is updating immediately, instead it becomes obvious that loading is occurring, with the loading indicator visible at the top of the page, before the new content finally pops into view.</p>

<p>We can simulate this experience in the local development environment with the <code>enableLatencySim</code> method on the <code>liveSocket</code> object directly in the browser console window:</p>

<pre><code class="js">&gt; liveSocket.enableLatencySim(500)
</code></pre>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937235-lv-transition-latency-compressed.mp4" type="video/mp4"></video>

<h2>Modal Transitions</h2>

<p>Notice that the UI is less abruptly updated when showing and hiding the modal form:</p>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937237-lv-transition-modal-transition-compressed.mp4" type="video/mp4"></video>

<p>Let’s take a closer look at how the modal is implemented to see if we can apply the same idea when navigating between pages. Here’s the outline of the markup generated by <code>mix phx.new</code> for the modal:</p>

<pre><code class="elixir">def modal(assigns) do
  ~H&quot;&quot;&quot;
  &lt;div id={@id} phx-mounted={@show &amp;&amp; show_modal(@id)} class=&quot;relative z-50 hidden&quot;&gt;
    &lt;div id={&quot;#{@id}-bg&quot;} class=&quot;fixed inset-0 bg-zinc-50/90 transition-opacity&quot; aria-hidden=&quot;true&quot; /&gt;
    &lt;div
      class=&quot;fixed inset-0 overflow-y-auto&quot;
      aria-labelledby={&quot;#{@id}-title&quot;}
      aria-describedby={&quot;#{@id}-description&quot;}
      role=&quot;dialog&quot;
      aria-modal=&quot;true&quot;
      tabindex=&quot;0&quot;
    &gt;
      &lt;div class=&quot;flex min-h-full items-center justify-center&quot;&gt;...
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  &quot;&quot;&quot;
end
</code></pre>

<p>The outer <code>&lt;div&gt;</code> has the <a href="https://tailwindcss.com/docs/display#hidden">hidden</a> tailwindcss utility class applied, which prevents the modal from being shown initially. The <code>phx-mounted</code> <a href="https://hexdocs.pm/phoenix_live_view/bindings.html#lifecycle-events">binding</a> invokes the JS command generated by <code>show_modal(@id)</code> to make the modal visible once the page is loaded and the LiveView web socket connected.</p>

<p>The <code>show_modal</code> function uses <a href="https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.JS.html#show/1">JS.show</a> to show the modal and fade in the background with a transition.</p>

<pre><code class="elixir">def show_modal(js \\ %JS{}, id) when is_binary(id) do
    js
    |&gt; JS.show(to: &quot;##{id}&quot;)
    |&gt; JS.show(
        to: &quot;##{id}-bg&quot;,
        transition: {&quot;transition-all transform ease-out duration-300&quot;, &quot;opacity-0&quot;, &quot;opacity-100&quot;}
    )
    |&gt; show(&quot;##{id}-container&quot;)
    |&gt; JS.focus_first(to: &quot;##{id}-content&quot;)
end
</code></pre>

<p>Another helper function <code>show/1</code> generated by <code>mix phx.new</code> is used from within <code>show_modal</code>:</p>

<pre><code class="elixir">def show(js \\ %JS{}, selector) do
  JS.show(js,
    to: selector,
    transition:
      {&quot;transition-all transform ease-out duration-300&quot;,
       &quot;opacity-0 translate-y-4 sm:translate-y-0 sm:scale-95&quot;,
       &quot;opacity-100 translate-y-0 sm:scale-100&quot;}
  )
end
</code></pre>

<p>The <a href="https://tailwindcss.com/docs/transition-property">tailwind css documentation</a> describes the effect of each class. </p>

<p>The general strategy is to initially render the elements as <code>hidden</code> or in some other initial state, then use <code>phx-mounted</code> to show the element and modify the classes applied.</p>

<p>TailwindCSS <code>transition</code> <code>transform</code>  <code>duration</code> <code>translate</code>  <code>scale</code> and <code>opacity</code> utility classes are used to define the transition.</p>

<h2>Fade in on mount</h2>

<p>The first transition we’ll apply is a fade-in as the page loads. This should slightly reduce the pop-in effect you experience after waiting for the loading to complete.</p>

<p>We’ll make an edit to the layout template in <code>lib/&lt;app_name&gt;_web/components/layouts/app.html.heex</code></p>

<pre><code class="jsx">// layouts/app.html.heex
&lt;main class=&quot;px-4 py-20 sm:px-6 lg:px-8 transition-all duration-500 opacity-0&quot; 
      phx-mounted={JS.remove_class(&quot;opacity-0&quot;)}&gt;
</code></pre>

<p>The initial render will include the tailwind css classes <code>transition-all duration-500 opacity-0</code> on the <code>&lt;main&gt;</code> element containing the LiveView content.
Once the <code>&lt;main&gt;</code> element is mounted and the web socket is connected, the <code>phx-mounted</code> <a href="https://hexdocs.pm/phoenix_live_view/bindings.html#lifecycle-events">binding</a> will execute the <code>JS.remove_class</code> command to remove the <code>&quot;opacity-0&quot;</code> class, allowing the content to fade in:</p>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937232-lv-transition-fade-in-on-mount-compressed.mp4" type="video/mp4"></video>

<h2>Fade out on load</h2>

<p>The next change is to detect when a new page is being loaded, and fade out the current content.</p>

<p>LiveView dispatches <a href="https://hexdocs.pm/phoenix_live_view/js-interop.html#live-navigation-events">custom window events</a> when page loading starts and finishes. These are used in <code>app.js</code> to show and hide the loading bar:</p>

<pre><code class="js">// Show progress bar on live navigation and form submits
topbar.config({ barColors: { 0: &quot;#29d&quot; }, shadowColor: &quot;rgba(0, 0, 0, .3)&quot; })
window.addEventListener(&quot;phx:page-loading-start&quot;, info =&gt; topbar.delayedShow(200))
window.addEventListener(&quot;phx:page-loading-stop&quot;, info =&gt; topbar.hide())
</code></pre>

<p>We can use the same events to add and remove a class from the <code>main</code> element during page loads:</p>

<pre><code class="js">// loading transitions
window.addEventListener(&quot;phx:page-loading-start&quot;, info =&gt; {
  if (info.detail.kind == &quot;redirect&quot;) {
    const main = document.querySelector(&quot;main&quot;);
    main.classList.add(&quot;phx-page-loading&quot;)
  }
})

window.addEventListener(&quot;phx:page-loading-stop&quot;, info =&gt; {
  const main = document.querySelector(&quot;main&quot;);
  main.classList.remove(&quot;phx-page-loading&quot;)
})
</code></pre>

<p>The <code>phx-page-loading</code> class is only applied when the event <code>detail.kind</code> property is <code>&quot; redirect&quot;</code> which indicates that a <code>&lt;.link navigate={}/&gt;</code> is being used to navigate to another page. This prevents the fade-out from being applied on simple <code>live_patch</code> updates, such as when closing a modal.</p>

<p>The <code>phx-page-loading</code> class will be defined by a <a href="https://tailwindcss.com/docs/plugins#adding-variants">custom tailwindcss variant</a>, allowing any utility classes to be conditionally applied while the page is loading.</p>

<p>The generated <code>tailwind.config.js</code> file already contains several custom variants used by LiveView to apply loading states to forms and buttons, we can add one more for <code>phx-page-loading</code></p>

<pre><code class="js">plugins: [
    require(&quot;@tailwindcss/forms&quot;),
    plugin(({addVariant}) =&gt; addVariant(&quot;phx-no-feedback&quot;, [&quot;.phx-no-feedback&amp;&quot;, &quot;.phx-no-feedback &amp;&quot;])),
    plugin(({addVariant}) =&gt; addVariant(&quot;phx-click-loading&quot;, [&quot;.phx-click-loading&amp;&quot;, &quot;.phx-click-loading &amp;&quot;])),
    plugin(({addVariant}) =&gt; addVariant(&quot;phx-submit-loading&quot;, [&quot;.phx-submit-loading&amp;&quot;, &quot;.phx-submit-loading &amp;&quot;])),
    plugin(({addVariant}) =&gt; addVariant(&quot;phx-change-loading&quot;, [&quot;.phx-change-loading&amp;&quot;, &quot;.phx-change-loading &amp;&quot;])),
    plugin(({addVariant}) =&gt; addVariant(&quot;phx-page-loading&quot;, [&quot;.phx-page-loading&amp;&quot;, &quot;.phx-page-loading &amp;&quot;]))
  ]
}
</code></pre>

<p>The syntax used to define the variant to allow it to be used on any element with the <code>phx-page-loading</code> class, or any element that is a descendant of one with the <code>phx-page-loading</code> class.</p>

<p>Let’s use <code>phx-page-loading:opacity-0</code> to fade out while the page is loading</p>

<pre><code class="jsx">&lt;main
  class=&quot;px-4 py-20 sm:px-6 lg:px-8 transition-all duration-500 opacity-0 phx-page-loading:opacity-0&quot;
  phx-mounted={JS.remove_class(&quot;opacity-0&quot;)}
&gt;
</code></pre>

<p>Now the page will fade out while loading, and fade back in once the new content is ready:</p>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937229-lv-blog-transitions-screen-recording-fade-out-in-compressed.mp4" type="video/mp4"></video>

<h2>Adding and Removing Table Rows</h2>

<p>Let’s have some fun with the table included in the generated <code>Index</code> LiveView.</p>

<p>The table component provides an <code>:action</code> slot used for the <code>Edit</code> and <code>Delete</code> links. In the <code>phx-click</code> binding for the Delete action, we <code>JS.push</code> a <code>&quot;delete&quot;</code> event to the LiveView, and apply some transition classes in a helper function <code>fade_away_left</code></p>

<pre><code class="elixir">&lt;:action :let={listing}&gt;
  &lt;.link
    phx-click={
      JS.push(&quot;delete&quot;, value: %{id: listing.id})
      |&gt; fade_away_left(&quot;#listing-#{listing.id}&quot;)
    }
    data-confirm=&quot;Are you sure?&quot;
  &gt;
    Delete
  &lt;/.link&gt;
&lt;/:action&gt;
</code></pre>

<p>Our <code>fade_away_left</code> helper will use the <code>JS.transition</code> function and some TailwindCSS transform classes to make it feel like the deleted item is sliding out of the table.</p>

<pre><code class="elixir">def fade_away_left(js \\ %JS{}, selector) do
  JS.transition(
    js,
    {&quot;transition-all transform ease-in duration-300&quot;, &quot;motion-safe:-translate-x-96 opacity-0&quot;, &quot;hidden&quot;},
    to: selector,
    time: 300
  )
end
</code></pre>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937214-blog-transitions-screen-recording-fade-away-left-compressed.mp4" type="video/mp4"></video>

<p>We can make new table rows slide in from the right, and smoothly scroll into view with some more transition classes and a <a href="https://hexdocs.pm/phoenix_live_view/js-interop.html#client-hooks-via-phx-hook">Phoenix Hook</a>:</p>

<pre><code class="elixir">&lt;tr
  :for={row &lt;- @rows}
  id={&quot;#{@id}-#{Phoenix.Param.to_param(row)}&quot;}
  class={[
    &quot;group hover:bg-zinc-50&quot;,
    if(@new_row == Phoenix.Param.to_param(row),
      do: &quot;transition-all transform duration-500 motion-safe:translate-x-96&quot;
    )
  ]}
  phx-mounted={
    if(@new_row == Phoenix.Param.to_param(row), 
      do: JS.remove_class(&quot;motion-safe:translate-x-96&quot;))
  }
  phx-hook={if(@new_row == Phoenix.Param.to_param(row), do: &quot;ScrollIntoView&quot;)}
&gt;
</code></pre>

<p>A new assign <code>@new_row</code> is added to conditionally add the transition classes to the <code>&lt;tr&gt;</code> element for the newly added row. Once mounted, the <code>motion-safe:translate-x-96</code> class is removed sliding the new row in from the right.</p>

<p>A <a href="https://hexdocs.pm/phoenix_live_view/js-interop.html#client-hooks-via-phx-hook">hook</a> is used to scroll the new row into view after a short delay when the table becomes large:</p>

<pre><code class="js">// app.js

let Hooks = {
  ScrollIntoView: {
    mounted() {
      setTimeout(() =&gt; this.el.scrollIntoView({behavior: &quot;smooth&quot;}), 500)
    }
  }
}
let csrfToken = document.querySelector(&quot;meta[name=&#39;csrf-token&#39;]&quot;).getAttribute(&quot;content&quot;)
let liveSocket = new LiveSocket(&quot;/live&quot;, Socket, { params: { _csrf_token: csrfToken }, hooks: Hooks })
</code></pre>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937225-blog-transitions-screen-recording-scroll-to-new-row-compressed.mp4" type="video/mp4"></video>

<h2>Accessibility and prefers-reduced-motion</h2>

<p>It’s important to respect the users preference for reduced motion by prefixing the any transform classes with the <a href="https://tailwindcss.com/docs/hover-focus-and-other-states#prefers-reduced-motion">motion-safe:</a> modifier. </p>

<p>Testing the app with reduced motion enabled in the system preferences confirms that we no longer have elements sliding across the page:</p>

<p><img src="https://www.datocms-assets.com/55475/1673940024-blog-transitions-screenshot-reduced-motion-setting-compressed.jpg" alt="Reduce motion setting on MacOS"></p>

<video autoplay muted loop playsinline src="https://www.datocms-assets.com/55475/1673937220-blog-transitions-screen-recording-reduced-motion-compressed.mp4" type="video/mp4"></video>

<p>See the <a href="https://tailwindcss.com/docs/hover-focus-and-other-states#prefers-reduced-motion">TailwindCSS docs</a>, <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion">MDN</a> and <a href="https://webkit.org/blog/7551/responsive-design-for-motion/">webkit blog</a> for more details.</p>

<h2>Wrapping Up</h2>

<p>I hope you find the techniques shown in this post useful to smooth out the navigation in your LiveView projects and liven things even further with some transitions.</p>


Let's improve the look and feel of our LiveView apps with page transitions that help users see what's happening when navigating between different parts of your application.

Improve UX with LiveView page transitions

Phoenix LiveView is a game changer for dynamic web application development. But what happens when we get serious traffic, and things slow down? What if we have some ambitious concurrency and latency requirements?

Monitoring Phoenix LiveView performance

<p>Numerical performance of Elixir code has received a massive boost with the release of Nx - Numerical Elixir.</p>

<p>Over the past few months José Valim has been dropping cryptic hints about an exciting, but mysterious new Elixir project called Nx. A series of tweets, alluding to big performance improvements got the community speculating as to what Nx could be.</p>

<p>The wait was finally over on Feb 9th, when on the <a href="https://thinkingelixir.com/podcast-episodes/034-jose-valim-reveals-project-nx/">Thinking Elixir</a> podcast, all was revealed.  Nx is Numerical Elixir, a tensor programming library for Elixir.  With Nx, we can develop tensor (multi-dimensional array) programs in Elixir and use a much faster backend to run them on the CPU or GPU. This is very exciting as it opens up a whole new set of use cases for Elixir in data science and machine learning and other numerically focussed workloads.</p>

<p>A new numerical function definition keyword, <code>defn</code>, is also introduced, wherein we can use a large subset of Elixir syntax to develop Nx programs.  Numerical definitions are compiled to computation graphs that describe the operations we want to perform, which are in turn compiled to efficient CPU or GPU code.  Early benchmarks show several orders of magnitude performance gains over plain Elixir, in some cases there are speed improvements of almost 5000x!</p>

<h3>Numbats and Numerical Notebooks?</h3>

<p>As a nice touch for us Australian alchemists, the mascot of the Nx project is the <a href="https://en.wikipedia.org/wiki/Numbat">numbat</a>, a native Australian marsupial.  The official Nx code repo on GitHub also has a cute <a href="https://github.com/elixir-nx/nx/raw/main/nx/nx.png">numbat logo</a>.</p>

<p>To see what all the fuss is about, Robert from Alembic (<a href="https://twitter.com/robertellen">@robertellen</a>) developed a code notebook that explores the Nx API, and presented the notebook at the Elixir Australia March meetup.</p>

<ul>
<li>The video of the talk is now up on <a href="https://www.youtube.com/watch?v=SgSbaGm5nR0">YouTube</a></li>
<li>The repository hosting the code notebook is available on <a href="https://github.com/team-alembic/elixir_nx_demo">Github</a>.</li>
</ul>

<p>You&#39;ll need to download the code notebook and run as a Phoenix LiveView application locally as this isn&#39;t protected against arbitrary code execution (so we can&#39;t host it yet). José has been recently hinting at the possibility of hosted notebooks though, so this may be possible soon? </p>

<h3>Tensors are multi dimensional arrays</h3>

<p>Nx, at its core, is tensor programming library.  In the context of Nx, tensors are multi-dimensional arrays.  For anyone familiar with <a href="https://www.tensorflow.org/">TensorFlow</a>, <a href="https://pytorch.org/">PyTorch</a>, or <a href="https://numpy.org/">NumPy</a>, tensors in Nx are equivalent to the tensors or multi-dimensional arrays in those environments.</p>

<p>To begin working with Nx, tensors are created from nested lists as well as binaries, and <code>Nx.reshape</code> allows a shape to be given to a flat binary.</p>

<pre><code class="elixir">iex(1)&gt; Nx.tensor([[1,2,3],[4,5,6]])
#Nx.Tensor&lt;
  s64[2][3]
  [
    [1, 2, 3],
    [4, 5, 6]
  ]
&gt;
</code></pre>

<pre><code class="elixir">iex(1)&gt; t = Nx.from_binary(&lt;&lt;1::integer,2::integer,3::integer,4::integer&gt;&gt;, {:u,8})
#Nx.Tensor&lt;
  u8[4]
  [1, 2, 3, 4]
&gt;
iex(2)&gt; t |&gt; Nx.reshape({2,2})
#Nx.Tensor&lt;
  u8[2][2]
  [
    [1, 2],
    [3, 4]
  ]
&gt;
</code></pre>

<p>Tensors can also be created from shapes (e.g. <code>{2,3}</code> for a 2-by-3 array) in a variety of ways with functions such as <code>Nx.iota</code> (creates an array of values incrementing from zero) or <code>Nx.broadcast</code> (adds dimensions to an array by duplicating the data in the array).</p>

<h3>Tensor-aware functions</h3>

<p>In Nx, we don&#39;t have to write loops over the elements of our tensors like we would if they were lists or maps.  A broad range of <em>tensor-aware</em> functions exist in the Nx library that will perform operations on tensors without explicit loops (maps or reduces).  This both makes the code terse and more performant.</p>

<p>Examples of tensor-aware functions are:</p>

<ul>
<li>unary functions such as <code>Nx.exp</code></li>
</ul>

<pre><code class="elixir">iex(16)&gt; [[1.0, 2.0],[3.0,4.0]] |&gt; Nx.tensor |&gt; Nx.exp
#Nx.Tensor&lt;
  f32[2][2]
  [
    [2.7182817459106445, 7.389056205749512],
    [20.08553695678711, 54.598148345947266]
  ]
&gt;
</code></pre>

<ul>
<li>binary functions such as <code>Nx.add</code></li>
</ul>

<pre><code class="elixir">iex(1)&gt; Nx.add(Nx.tensor([1,2,3,4]), Nx.tensor([5,6,7,8]))
#Nx.Tensor&lt;
  s64[4]
  [6, 8, 10, 12]
&gt;
</code></pre>

<ul>
<li>and, aggregate functions such as <code>Nx.sum</code></li>
</ul>

<pre><code class="elixir">iex(1)&gt; [[1.0,2.0],[3.0,4.0]] |&gt; Nx.tensor |&gt; Nx.sum
#Nx.Tensor&lt;
  f32
  10.0
&gt;
</code></pre>

<p>The tensor-aware operations are the key to the potential speed-ups that Nx provides.  Each backend for Nx implements efficient versions of the operations that execute in parallel.</p>

<p>Further to an API of tensor operations, Nx introduces Numerical Definitions to make developing tensor code more natural.  By defining a function with <code>defn</code> instead of the usual <code>def</code>, we can write tensor code using a broad subset of Elixir syntax, including tensor-aware version of operators we know from <code>Kernel</code>, such as <code>+</code>, <code>-</code>, <code>*</code>, and <code>/</code>.  The canonical example of using numerical definitions is the <code>softmax</code> function:</p>

<pre><code class="elixir">defmodule Formula do
  import Nx.Defn

  defn softmax(t) do
    Nx.exp(t) / Nx.sum(Nx.exp(t))
  end
end
</code></pre>

<h3>Easily Configurable Backends</h3>

<p>Currently, the Nx contributors have implemented backends for Nx in EXLA and Torchx, which are Elixir clients for Google&#39;s XLA (Accelerated Linear Algebra), and LibTorch, respectively.  Both support CPU or GPU-based acceleration of Nx functions and numerical definitions, and below we can see how easy it is to specify different backend configurations globally or even per tensor.</p>

<pre><code class="elixir">Nx.tensor([1,2,3,4], backend: Torchx.Backend)

@default_defn_compiler {EXLA, run_options: [keep_on_device: true]}
</code></pre>

<h3>Benchmarks show 5000x speed boost!</h3>

<p>The Nx <a href="https://github.com/elixir-nx/nx">GitHub repo</a> presents a nice benchmark for the softmax numerical definition:</p>

<pre><code class="graphql">Comparison:
xla gpu f32 keep      15308.14
xla gpu f64 keep       4550.59 - 3.36x slower +0.154 ms
xla cpu f32             434.21 - 35.26x slower +2.24 ms
xla gpu f32             398.45 - 38.42x slower +2.44 ms
xla gpu f64             190.27 - 80.46x slower +5.19 ms
xla cpu f64             168.25 - 90.98x slower +5.88 ms
elixir f32                3.22 - 4760.93x slower +310.94 ms
elixir f64                3.11 - 4924.56x slower +321.63 ms
</code></pre>

<p>The takeaway that using 32-bit floating point tensors, keeping the tensors on the GPU memory produces massive performance gains.</p>

<p>Nx represents a new frontier for the Elixir ecosystem.  Adding high performance numerical computation to Elixir allows us to solve more problems on the BEAM and will hopefully make Elixir an attractive choice of language in more domains.  We commend the Nx contributors for such a great initiative.  We&#39;re looking forward to seeing where Nx takes Elixir and the BEAM.</p>

<p><a href="https://twitter.com/ElixirSydney/status/1373203685006712837">https://twitter.com/ElixirSydney/status/1373203685006712837</a></p>

<h2>Elixir Australia</h2>

<p>Elixir Australia is a joint effort by <a href="https://www.meetup.com/elixir-sydney/">Elixir Sydney</a> (sponsored by Alembic) and <a href="https://www.meetup.com/Elixir-Melbourne/">Elixir Melbourne</a> to host monthly online meetups for the Elixir community.  We encourage all Elixir alchemists globally to join either of the meetup groups to keep an eye out for upcoming events (the next scheduled meetup is Wednesday, April 21, 2021 7:00 PM to 9:00 PM GMT+10).  The <a href="https://www.youtube.com/c/ElixirAustralia">YouTube channel</a> can also be subscribed to to keep up to date with videos from the meetups.</p>

<p>Please check out the links and we hope to see you at an Elixir Australia meetup soon!</p>

<h2>Links</h2>

<ul>
<li>The official Elixir Nx code repository: <a href="https://github.com/elixir-nx/nx">https://github.com/elixir-nx/nx</a></li>
<li>Kicking the tyres on Nx talk: <a href="https://www.youtube.com/watch?v=SgSbaGm5nR0">https://www.youtube.com/watch?v=SgSbaGm5nR0</a></li>
<li>Code repo for the code notebook: <a href="https://github.com/team-alembic/elixir_nx_demo">https://github.com/team-alembic/elixir<em>nx</em>demo</a></li>
<li>Elixir Sydney: <a href="https://www.meetup.com/elixir-sydney/">https://www.meetup.com/elixir-sydney/</a></li>
<li>Elixir Melbourne: <a href="https://www.meetup.com/Elixir-Melbourne/">https://www.meetup.com/Elixir-Melbourne/</a></li>
<li>Elixir Australia Youtube channel: <a href="https://www.youtube.com/c/ElixirAustralia">https://www.youtube.com/c/ElixirAustralia</a></li>
</ul>


Numerical performance of Elixir code has received a massive boost with the release of Nx - Numerical Elixir. This is an exciting addition to the Elixir ecosystem, read on to find out what you can do with it!



High performance Numerical Elixir with Nx

<p>This article is part of a series about functional programming with FP-TS. It&#39;s aimed at those who may have heard good things about functional programming but don&#39;t really know where to start approaching the subject. More specifically it&#39;s for JavaScript users who may be aware that their language has something to do with functional programming but not really sure how. Let&#39;s get started!</p>

<h2>What is FP-TS?</h2>

<p>By some definition, one might call JavaScript a functional language due to the fact it has first class functions. However, ask a functional programmer and they&#39;ll most likely balk at such a description as functional programming is much broader in scope.</p>

<p>TypeScript does begin to address these discrepancies by including types. While this is an important start, it really is only a start.</p>

<p><code>fp-ts</code> is an additional layer on top of TypeScript, which leverages existing capabilities and gives us the full power of the functional paradigm. More precisely, it is a TypeScript library which provides additional methods to capture all the concepts a functional programmer would need.</p>

<h2>What are pipelines?</h2>

<p>One of the primary philosophies of functional programming is that you don&#39;t mutate data. Instead the idea is that you have an input which is &quot;piped&quot; through a function. The important point here is no data is mutated; the source of the input is the unaffected by the function and the output is new data. </p>

<p>You can connect these &quot;pipes&quot; together into a longer pipe, much like how actual physical pipes are put together. We could write this as:</p>

<pre><code class="tsx">input -&gt; f1 -&gt; output1 -&gt; f2 -&gt; output2 -&gt; f3
</code></pre>

<p>If you have ever seen any Elixir, F# or Elm code one might see this written with the following syntax:</p>

<pre><code class="elixir">input |&gt; f1 |&gt; f2 |&gt; f3
</code></pre>

<p>If you have used Unix pipes then it&#39;s also the same thing, except with lines of text piped from stdin to stdout / stderr.</p>

<pre><code class="bash">cat file.txt | wc -l
</code></pre>

<h2>Why are pipelines useful?</h2>

<p>So why is this useful? That is a good question. After all, even spaghetti code can technically perform any task we might want. The answer lies in the fact that there is more to writing code than simply getting it to compile. For some the aesthetics of the good code is reward enough, but for others who are more practically minded and aren&#39;t interested in becoming one with the code, there are other incentives. </p>

<p>First of all if we build our programs out of functions without side-effects, then reasoning about your code becomes much simpler. We can be confident that we aren&#39;t going to get nasty side effects from unintentionally mutating data which other code may be using.</p>

<p>Secondly, it allows us to read and write our code as a series of self contained functions which reads more easily; no parsing of complicated &#39;for&#39; loops and &#39;if&#39; statements is required to see what is happening  at the top level of the pipeline. If you need to see the detail, then you can inspect each function in the pipeline in isolation.</p>

<p>If we wanted to try and achieve the same effect in TypeScript where we pass the value of one function to another, the syntax forces us to write something awkward like:</p>

<pre><code class="tsx">output = f3(f2(f1(input)))
</code></pre>

<h2>The Pipe operator in FP-TS</h2>

<p><code>fp-ts</code> gives us the pipe operator to create a pipeline of function:</p>

<pre><code class="tsx">import { pipe } from &#39;fp-ts/lib/function&#39;

const f1 = (x) =&gt; x + 1
const f2 = (x) =&gt; x + 2
const f3 = (x) =&gt; x + 3

pipe(0, f1, f2, f3)  // returns 6
</code></pre>

<p>When we use this in a real program this let&#39;s us reason about high level program flow more easily:</p>

<pre><code class="tsx">pipe(query, fetchFromDatastore, transformResult)
</code></pre>

<p>You can find some more examples in the <a href="https://gcanti.github.io/fp-ts/modules/function.ts.html#pipe">FP-TS pipe documentation</a>.</p>

<h2>The Flow operator in FP-TS</h2>

<p><code>fp-ts</code> gives us another very similar tool called the <code>flow</code> operator. It&#39;s very similar to <code>pipe</code> except that all the arguments are functions and the output it returns is itself a function which one <strong>then</strong> passes in the initial input.</p>

<p>More concretely:</p>

<pre><code class="tsx">import { flow } from &#39;fp-ts/lib/function&#39;

const splitBy = (char: string) =&gt; (s: string) =&gt; s.split(char)
const count = (xs: unknown[]) =&gt; xs.length
const splitBySpace = splitBy(&#39; &#39;)
const countWords = flow(
  splitBySpace,
  count
)

countWords(&#39;a b c&#39;)  // returns 3
</code></pre>

<p>The advantage of <code>flow</code> is that if you want to pass the result of a series of piped functions into another function, you don&#39;t need write an anonymous function.</p>

<p>Instead of:</p>

<pre><code class="tsx">someOtherFunction = (num: number, func: (number) =&gt; number)
someOtherFunction(1, (num) =&gt; pipe(num, f1, f2, f3))
</code></pre>

<p>We can do:</p>

<pre><code class="tsx">someOtherFunction = (num: number, func: (number) =&gt; number)
someOtherFunction(1, flow(f1, f2, f3))
</code></pre>

<p>You can find some more examples in the <a href="https://gcanti.github.io/fp-ts/modules/function.ts.html#flow">FP-TS flow documentation</a>.</p>

<h2>Currying with FP-TS</h2>

<p>What if we want to add functions with multiple arguments to a pipeline? This won&#39;t work because <code>pipe</code> is expecting a single argument function. There is a technique which we can take advantage of to help us.</p>

<p>Currying is the conversion of a function with multiple arguments into a series of single argument functions. For example, a 3 argument function <code>f(x, y, z)</code> becomes <code>((g(x))(y))(z)</code> or <code>g(x)(y)(z)</code>. Each new single argument function returns a new function which again takes the next argument until last function just returns the result.</p>

<p>So instead of:</p>

<pre><code class="tsx">declare const f: (x: A, y: B, z: C) =&gt; D
</code></pre>

<p>We can define a new curried version:</p>

<pre><code class="tsx">declare const curriedF = (x: A) =&gt; (y: B) =&gt; (z: C) =&gt; { return f(x, y, z) }
</code></pre>

<p>So that:</p>

<pre><code class="tsx">const first: (y: B) =&gt; (z: C) =&gt; D = curriedF(a)
const second: (z: C) =&gt; D          = curriedF(a)(b)
const returnValue: D               = curriedF(a)(b)(c)
</code></pre>

<p>This resembles piping but instead of passing the output of a function to the next function, we can think of it as sending a function to a value. In fact if we had a function which takes values and returns a function which takes a curried function and returns the next curried function in the sequence, we could use piping to get this behaviour. As it so happens, <code>fp-ts</code> provides just such a function, <code>ap</code> which is short for &quot;apply&quot;.</p>

<pre><code class="tsx">declare const ap: &lt;A&gt;(arg: A) =&gt; &lt;B&gt;(f: (arg:A) =&gt; B) =&gt; B

ap :: a -&gt; (a -&gt; b) -&gt; b 
</code></pre>

<p>This means we can combine this with piping to get the following:</p>

<pre><code class="tsx">const returnValue = pipe(curriedF, ap(a), ap(b), ap(c))
</code></pre>

<p>What happens here is that we pass the curried version of our function <code>f()</code> along the pipeline passing in values one by one. This might seem a little convoluted In practice we would probably use sequences to do this kind of thing rather than <code>ap</code>, however that&#39;s a topic for another post.</p>

<h2>Conclusion</h2>

<p>We&#39;ve really only just scratched the surface of what <code>fp-ts</code> is about, but hopefully this article has provided some insight into functional programming techniques and the clarity it can bring to your code. In particular the <code>pipe</code> and <code>flow</code> operator gives us a way to express our program in a more linear way.</p>

<p>This is just the beginning however and we shall dive deeper into <code>fp-ts</code> in later articles. So stay tuned!</p>

<h2>References</h2>

<ul>
<li><a href="https://gcanti.github.io/fp-ts/">FP-TS Documentation</a></li>
<li><a href="https://github.com/gcanti/fp-ts">FP-TS GitHub Repo</a></li>
</ul>


FP-TS is a useful library for programming TypeScript with the full power of functional programming. It provides some very handy functional tools from Pipelines to Currying to make our TypeScript programs more elegant and robust.

Intro to Functional Programming in TypeScript with FP-TS

<p>A git commit is a snapshot of the repo at a particular point in time. They&#39;re used to save and track changes in a codebase, and are the building blocks of Git. At a high level at least, at a more fundamental level <a href="https://git-scm.com/book/en/v2/Git-Internals-Git-Objects">everything in Git is stored as an Object</a>. Learning how commits work is fundamental to understanding how to take advantage of Git&#39;s full potential.</p>

<h2>What is in a Git commit?</h2>

<p>A Git commit object contains all the metadata of the commit. This includes the date, author, committer, the commit message, as well as the directory tree object hash, and parent commit hash(es).</p>

<p>To see the contents of the most recent commit and it&#39;s metadata on the current working branch, run <code>git show --pretty=raw</code>. It&#39;ll look something like this:</p>

<pre><code class="bash">commit 4a07d916c1802d606a41e00b52e2068f6510ded6
tree 7ed44fc18fc76c66631ac3d4169107b1a48f9c58
parent a08bc7c445e17faad93000089aef41c71f129ade
author Jane Smith &lt;jane@alembic.com.au&gt; 1636455211 +1100
committer Jane Smith &lt;jane@alembic.com.au&gt; 1636455211 +1100

example commit

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e69de29

&lt;...the rest of the diff...&gt;
</code></pre>

<p>You can see all the metadata we mentioned above. The author, date and commit message are obvious but the <code>commit</code>, <code>tree</code> and <code>parent</code> hashes are worth explaining in more detail.</p>

<p>The hash of a commit&#39;s parent commit acts as a pointer linking the all commits together. A commit can have multiple parent commits if a merge of one or more branches has been made.</p>

<p>The tree object referenced in the above commit contains the directory listing for the commit. This is made up of other branching Tree objects (sub-directories) and Blob objects (Binary large object containing the contents of the file). You can <a href="https://git-scm.com/book/en/v2/Git-Internals-Git-Objects">read more about Tree and Blob objects here.</a></p>

<h2>What if I amend a commit?</h2>

<p>Git commits are immutable. If you amend a commit and make changes, Git will create a completely new commit and a new commit identifier. It doesn&#39;t matter what metadata you change, the identifier will change. The old commit with it&#39;s original identifier still exists, and although it may not be visible we&#39;ll show you how to find it later on in this article.</p>

<p>Commits don&#39;t reference any of their children, so any changes to the commit&#39;s children won&#39;t alter the parent. Any changes to a parent commit however, will result in a new commit identifier for each of its children. This will be important to remember when we introduce Git rebase. </p>

<h2>How do hash functions work?</h2>

<p>A hash function is an algorithm that takes in data and returns a unique value. The most important features of a hash function are:</p>

<ul>
<li>A hash function will always produce the same output for a given input</li>
<li>Any small change in the input data will produce a completely different output</li>
<li>The hash function is not reversible.</li>
</ul>

<p>The unique 40-character identifiers of a Git commit are SHA-1 hashes. For more details, check out <a href="https://brilliant.org/wiki/secure-hashing-algorithms/">how hashing works and specifically SHA-1</a>.</p>

<h3>Small changes in input change the output a lot</h3>

<p>Let&#39;s test this in our terminal:</p>

<pre><code class="bash">$ echo -n &quot;abcd&quot; | openssl sha1
81fe8bfe87576c3ecb22426f8e57847382917acf

$ echo -n &quot;abcd&quot; | openssl sha1
81fe8bfe87576c3ecb22426f8e57847382917acf
</code></pre>

<p>The input <code>&quot;abcd&quot;</code> will return a hash of <code>81fe8bfe87576c3ecb22426f8e57847382917acf</code> regardless of how many times it is run, or what system it is run on. If you change the text, even slightly, the returned hash will be wildly different. In the following cases we only change the input <code>&quot;abcd&quot;</code> by 1 single bit. Notice how different the resulting hash is?</p>

<pre><code class="bash">$ echo -n &quot;abcd&quot; | openssl sha1
81fe8bfe87576c3ecb22426f8e57847382917acf

$ echo -n &quot;bbcd&quot; | openssl sha1
3c1238dc8e11a8de5af8e80e3d19f9c4b53b629e

$ echo -n &quot;abce&quot; | openssl sha1
0a431a7631cabf6b11b984a943127b5e0aa9d687
</code></pre>

<p>This means that if one change is made to a file, directory or the commit&#39;s metadata, no matter how small that change is, you will receive an entirely new (and very different) hash identifier.</p>

<h3>Hash functions are irreversible</h3>

<p>Hash functions are irreversible, so there is no practical way to find the contents that made the hash identifier. If this was possible it would break the cryptographic security of git. It is very unlikely but possible for a SHA-1 collision to occur, where different contents produce the same identifier.</p>

<blockquote>
<p>Two objects colliding accidentally is exceedingly unlikely. If you had five million programmers each generating one commit per second, your chances of generating a single accidental collision before the Sun turns into a red giant and engulfs the Earth is about 50%.</p>
</blockquote>

<p>However unlikely, it could be used as part of a collision attack. In 2017 <a href="https://github.blog/2017-03-20-sha-1-collision-detection-on-github-com/">the first SHA-1 collision was detected on GitHub</a>, which is an interesting read. TL;DR - Git is not likely to be exploited in this way anytime soon, for reasons explained in the article.</p>

<h2>Everything you commit to git is safe</h2>

<p>Remember that Git commits are immutable. Git never really deletes anything that&#39;s been committed, unless you explicitly delete it. Even when performing operations such as rebasing or commit amending that modify commits. When we change a commit in this way, they are not actually deleted, a new commit is made and the old commit continues to be stored by Git even though it may not be visible through ordinary commands.</p>

<p>When a commit has nothing referencing it (a branch for example), it becomes invisible (to commands like <code>git log</code>). Git hides these from you because otherwise it would very hard to navigate around the mess of old commits. However the old commits still exist.</p>

<p>It&#39;s important to remember that you can always reference any commit by its SHA-1 hash. Remembering 40 character hashes is probably beyond the capabilities of most mortals, so one very handy way that we can find these old, invisible commits is in the <code>reflog</code>.</p>

<h2>Finding invisible commits with git reflog</h2>

<p>References (or &quot;refs&quot;) can be thought of as <a href="https://git-scm.com/book/en/v2/Git-Internals-Git-References">named pointers to commits</a>. Reference logs, or &quot;ref logs&quot; represent the history of every commit that git has ever pointed to. Any time the the tip of branches are changed it&#39;s recorded in the <code>reflog</code>. This could be through switching branches, merges, or even a rebase or amendment. This means that if you accidentally &quot;lose&quot; some data, there&#39;s a very good chance it is in the reflog, and you can retrieve it from there.</p>

<p><code>git reflog</code> will show the reflog for the current working directory. By default, <code>git reflog</code> will output the reflog of the <code>HEAD</code> ref. You can get a complete reflog of all refs by executing <code>git reflog show --all</code>.</p>

<h2>Summary</h2>

<p>A Git commit contains all of the commit metadata and a snapshot of the repo at that point in time. The unique 40 character hash that represents the commit allows Git to track your commits and changes to the repo over time. They cannot be reverse engineered to see their contents.</p>

<p>When a commit is modified with <code>git commit --amend</code> or <code>git rebase</code>, it is not deleted, only replaced with a new commit with a new identity. We can always retrieve commits that are no longer visible via the reflog. Knowing the hash of the commit will make it easy to undo almost any git command with <code>git reset --hard</code>.</p>

<p>Understanding Git fundamentals and what&#39;s happening behind the curtain of Git will quickly increase your confidence and abilities, allowing you to take full advantage of Git and its tools.</p>

<p>Stay tuned as we&#39;ll be publishing a series of articles about understanding Git fundamentals and improving your Git Fu.</p>


Ever wondered how git commits work? This article explains what makes up the identity of a git commit. Knowing this will help you become much more confident running advanced git commands like rebase.

The identity of a Git commit explained

<p>This article is part of a series aimed at helping TypeScript users unfamiliar with functional programming approach it by examining the <code>fp-ts</code> library.  </p>

<p>Someone new to functional programming who read the <a href="https://alembic.com.au/blog/intro-to-functional-programming-in-typescript-with-fp-ts">previous article on pipe, flow and currying</a> might wonder how pipelines are useful with real life scenarios. For example what happens if we have to deal with something which sometimes returns no value? To address this problem we will introduce the <code>Option</code> monad. </p>

<p>To make this as easy to digest as possible, we will avoid any subsequent mention of the dreaded monads, and simply describe the <code>Option</code> in a practical way. In fact you can just think of <code>Option</code> as a module. The <code>Option</code> module defines its own type, which is unsurprisingly named <code>Option</code>. </p>

<h2>Discriminated Unions in TypeScript</h2>

<p>If you are reading this article it probably means you are familiar with how types can be built from other types in TypeScript, in particular using object types and with the operators for unions <code>|</code>. </p>

<p>For example, from a type <code>A</code> we can create a new object type <code>objectA</code> which takes the form: </p>

<pre><code class="tsx">{
    _tag: &#39;A&#39;,
    value: valueOfTypeA
}
</code></pre>

<p>I can create another type of an analogous form from type <code>B</code> called <code>objectB</code>:</p>

<pre><code class="tsx">{
    _tag: &#39;B&#39;,
    value: valueOfTypeB
}
</code></pre>

<p>I could then define a new type from these two <code>C = objectA | objectB</code>. This means that an element of type <code>C</code> will be of either type <code>objectA</code> or <code>objectB</code>.</p>

<p>Our new type <code>C</code> is a demonstration of what is called a discriminated union. The &quot;union&quot; part of the term should be fairly obvious, given that it is a union of two types. The &quot;discriminated&quot; part of the term has to do with the <code>_tag</code> property. An element of type <code>objectA</code> can conceptually be thought as an element of type <code>A</code>  which has been tagged.</p>

<p>In TypeScript, <strong>discriminated unions can be implemented as unions tagged with a string field</strong>, and they are needed to implement the Option type<strong>.</strong> Read more about <a href="https://www.typescriptlang.org/docs/handbook/2/narrowing.html#discriminated-unions">discriminated unions in the TypeScript documentation</a>. </p>

<h2>Defining Options</h2>

<h3>Defining the Option type</h3>

<p>The <code>Option</code> module can be imported directly</p>

<pre><code class="tsx">import * as O from &quot;fp-ts/lib/Option&quot;;
</code></pre>

<p>As mentioned we can import the <code>Option</code> type from this module. If we were to define it ourselves it would look like:</p>

<pre><code class="tsx">type Option&lt;A&gt; = None | Some&lt;A&gt;
</code></pre>

<p>This means that an element of type <code>Option</code> will either be of type <code>Some&lt;A&gt;</code> which looks like:</p>

<pre><code class="tsx">{ _tag: &#39;Some&#39;, value: someValueOfTypeA }
</code></pre>

<p>or of type <code>None</code> which looks like:</p>

<pre><code class="tsx">{ _tag: &#39;None&#39; }
</code></pre>

<p>This looks a lot like a discriminated union except one of the types is <code>void</code>. The <code>Some&lt;A&gt;</code> value can be thought of a value of type <code>A</code> tagged with <code>Some</code>, while <code>None</code> is just the tag <code>None</code>.</p>

<p>We can already get a sense of how this will be useful. Our first pass of our code might follow a naive approach which while in a big picture sense does what we want, in practice breaks down because some part of the process might not return a value. This would be a problem, given we&#39;re advocating functional programming where one wants continuity of input and output through a series of functions.</p>

<p>The <code>Option</code> module remedies this problem by taking those situations where a return type is <code>void</code> and turning it into an actual element of type <code>None</code>. Otherwise when we do get a value, say of type <code>A</code>, that value will be tagged with <code>Some</code> and returned, or to be more accurate an element of type <code>Some&lt;A&gt;</code> with the value inside will be returned.</p>

<h3>Pushing a value into Option world</h3>

<p>We can only take advantage of this <code>Option</code> type if we have a way to somehow translate the values and methods in our naive code into an &#39;option-ized&#39; version. Thankfully the <code>Option</code> module provides what we need.</p>

<p>First things first, we&#39;ll see how we can take elements of some type and bring them into the world of <code>Option</code>. </p>

<p>The first one is <code>of</code></p>

<pre><code class="tsx">declare const of: &lt;A&gt;(a: A) =&gt; Option&lt;A&gt;
</code></pre>

<p><code>of</code> takes in a value and returns an option in a natural way </p>

<pre><code class="tsx">O.of(aValueOfTypeA)           // returns { _tag: &#39;Some&#39;, value: aValueOfTypeA }
</code></pre>

<p>There is another method <code>fromNullable</code> which has the same effect provided the value being entered isn&#39;t <code>null</code> or <code>undefined</code>.</p>

<pre><code class="tsx">O.fromNullable(aValueOfTypeA)   // returns { _tag: &#39;Some&#39;, value: aValueOfTypeA }
</code></pre>

<p>However, if it is of a None type, the results are different</p>

<pre><code class="tsx">O.of(undefined)           // returns { _tag: &#39;Some&#39;, value: undefined }
O.fromNullable(undefined)   // returns { _tag: &#39;None&#39; }
</code></pre>

<p>To generate a None Option one can simply use <code>none</code>, for example:</p>

<pre><code class="tsx">O.none     // returns { _tag: &#39;None&#39; }
</code></pre>

<h3>How to move around in Option world</h3>

<p>There is nothing stopping one from using repeated applications of the <code>some</code> function creating a kind of &quot;nested&quot; <code>Option</code>.</p>

<p>For example:</p>

<pre><code class="tsx">const someOption1 = O.some(1)            // { _tag: &#39;Some&#39;, value: 1 }
const someOption2 = O.some(someOption1)  // { _tag: &#39;Some&#39;, value: { _tag: &#39;Some&#39;, value: 1 }}
</code></pre>

<p>This isn&#39;t particularly useful. If we wanted to get back down to a &quot;lower level&quot; we can use <code>flatten</code>, like so:</p>

<pre><code class="tsx">const someOption2 = { _tag: &#39;Some&#39;, value: { _tag: &#39;Some&#39;, value: 1 } }

O.flatten(someOption) // returns { _tag: &#39;Some&#39;, value: 1 }
</code></pre>

<p>But it&#39;s not enough just to be able to turn the values we have into options. The functions found in our original code don&#39;t know about options so they are useless here. Worse yet there is still the issue that these functions might possibly not even give a value at all. Thankfully the <code>Option</code> module gives us a way to &quot;lift up&quot; a function which so that we can deal with this problem.</p>

<p><code>map</code> is what is known as a higher order function, for the reason that it takes a function/s as an argument, and returns a function. It takes in a function of type <code>(arg: A): B</code> and spits out a new function of type <code>(arg: Option&lt;A&gt;): Option&lt;B&gt;</code>.</p>

<p>Let&#39;s give a concrete example:</p>

<p>We use the inbuilt <code>fp-ts</code> tool, <code>fromNullable</code> to create an item of type <code>Option&lt;string&gt;</code>:</p>

<pre><code class="tsx">const optOfStr: O.Option&lt;string&gt; = O.some(&quot;asd&quot;) // { _tag: &#39;Some&#39;, value: &#39;asd&#39; }
</code></pre>

<p>We define a function which takes in an item of type string and returns a value of type number:</p>

<pre><code class="tsx">const strLenPlus1:((str: string) =&gt; number) = ((str: string) =&gt; (str.length + 1))
strLenPlus1(&quot;asd&quot;)   // returns 4
</code></pre>

<p>We &quot;Option-ize&quot; this new function with <code>map</code>, creating a function which takes in an item of type <code>Option&lt;string&gt;</code> and returns a value of type <code>Option&lt;number&gt;</code>:</p>

<pre><code class="tsx">const optStrLenPlus1:((str: O.Option&lt;string&gt;) =&gt; O.Option&lt;number&gt;) 
= O.map(strLenPlus1)

optStrLenPlus1(optOfStr)    // returns { _tag: &#39;Some&#39;, value: 4 }
</code></pre>

<p><code>chain</code> is another tool provided by <code>fp-ts</code> to create new versions of functions but which is more powerful than <code>map</code>. It is also a higher order function, but the input is a function of the form <code>(arg: A): Option&lt;B&gt;</code> and returns a function of the form <code>(arg: Option&lt;A&gt;): Option&lt;B&gt;</code>.</p>

<p>Now lets give a concrete example of <code>chain</code> :</p>

<p>If we start with a function that takes in a string and returns an <code>Option&lt;number&gt;</code></p>

<pre><code class="tsx">const stringLengthOption = (str: string) =&gt; O.some(str.length)
stringLengthOption(&quot;asdf&quot;)   // { _tag: &#39;Some&#39;, value: 4 }
</code></pre>

<p>We can now make a new function which takes in an <code>Option&lt;string&gt;</code> and returns <code>Option&lt;number&gt;</code></p>

<pre><code class="tsx">const stringOptionToNumberOption =  O.chain(stringLengthOption)
const someStringOption = O.some(&quot;asdf&quot;)   // { _tag: &#39;Some&#39;, value: &#39;asdf&#39; }
stringOptionToNumberOption(someStringOption)   // { _tag: &#39;Some&#39;, value: 4 }
</code></pre>

<h2>Conclusion</h2>

<p>So even in the case of where we aren&#39;t guaranteed values of a non-nullable type or even any value at all, we don&#39;t have to throw in the functional programming towel. The <code>fp-ts</code> library gives us tools to enhance what TypeScript already gives us so that we don&#39;t have to break from the function to function pattern of our program. In future articles we&#39;ll see how the <code>Option</code> module is just an introduction to the way <code>fp-ts</code> helps us avoid these kind of problems.</p>

<h2>References</h2>

<ul>
<li><a href="https://alembic.com.au/blog/intro-to-functional-programming-in-typescript-with-fp-ts">Intro to Functional Programming in TypeScript with FP-TS</a></li>
<li><a href="https://gcanti.github.io/fp-ts/modules/Option.ts.html">FP-TS Option docs</a></li>
<li><a href="https://www.typescriptlang.org/docs/handbook/2/narrowing.html#discriminated-unions">Discriminated Unions in TypeScript</a>. </li>
<li><a href="https://gcanti.github.io/fp-ts/">FP-TS Documentation</a></li>
<li><a href="https://github.com/gcanti/fp-ts">FP-TS GitHub Repo</a></li>
</ul>


FP-TS is a useful library for programming TypeScript with the full power of functional programming. In this article we explore the FP-TS Option feature and how Discriminated Unions help implement it.

Exploring FP-TS Options in TypeScript

<p>We love <a href="/services/elixir-development">Elixir</a>, <a href="/services/typescript-development">TypeScript</a>, <a href="/services/graphql-consulting">GraphQL</a> and <a href="/services/react-development">React</a> and these are some of the exciting topics we&#39;ve been exploring and experimenting with.</p>
