Watch a process using Turbo streams

After upgrading to Hotwire, I wanted to try out Turbo Streams. In CloudSh there is a background job that runs a Golang application to index sites. That seemed like a cool thing to use Turbo Streams so I can see the console output.

Turbo Streams just work. The basic flow is capture the process output in the background job, broadcast it to the UI, and follow along by scolling as new data comes in.

Turbo Streams

First I needed a new page to "watch" the output. I added a watch action and view.

class IndicesController < ApplicationController
  # ...

  def watch
  end
end

The view setups up the Turbo Stream and a container for the log lines.

= turbo_stream_from @index

div#log_lines class="h-full overflow-auto bg-gray-900 text-white p-4 text-sm font-mono border-2 border-black" style="height: 50vh"

Next an ActiveModel is needed to broadcast, specifically with the Turbo::Broadcastable concern. I don't want this data stored in the database to I created a model to hold a line of output.

class LogLine
  include ActiveModel::Model
  include ActiveModel::Serialization
  include ActiveModel::Attributes
  include ActiveModel::AttributeMethods
  include Turbo::Broadcastable
  extend ActiveModel::Naming

  attribute :line

  alias_method :to_hash, :serializable_hash

  def persisted?
    false
  end

  def id
    nil
  end

  def to_s
    line
  end

  def to_html
    line
  end

  def broadcast index
    broadcast_append_to index
  end
end

Turbo Streams will look for a view partial to render the broadcasts using the model being broadcast.

# app/views/log_lines/_log_line.html.slim
p = log_line.to_html

With this I can create a LogLine and broadcast it to the watch page for the Index. It can even be run from the rails console.

LogLine.new(line: "Test line").broadcast index

Watching the process

In the background job, I needed to capture the STDOUT and broadcast each line. Popen2e provides this.

    # ...
      cmd = "cloudsh index #{index.domain} -d 100 -l #{limit} -x"

      status = Open3.popen2e(cmd) { |stdin, stdout_and_stderr, wait_thr|
        stdout_and_stderr.each { |line|
          LogLine.new(line: line).broadcast index
        }
        wait_thr.value
      }

      LogLine.new(line: "Completed: exit: #{status.to_i}").broadcast index
    # ...

Now after queuing the background job I redirect to the watch action created earlier.

Scrolling

With data streaming to the UI the problem became, the line would append, but not scroll into view. Making it very hard to watch the process.

The other side of Hotwire is Stimulus, so I created a controller and tried to hook into the turbo:before-stream-render. An after stream render event would have been better, but that's not an event in Turbo. Even so, I couldn't get the turbo:before-stream-render to fire.

I came accoss this Event to know a turbo-stream has been rendered discussion which suggests using MutationObserver. This was new to me, but integrates with Stimulus nicely.

Need to add the controller to the watch page on the container div.

  div#log_lines class="h-full overflow-auto bg-gray-900 text-white p-4 text-sm font-mono border-2 border-black" style="height: 50vh" data-controller="scroll"

The controller listens for childList changes, finds the last child, and scrolls it into view.

// app/javascript/controllers/scroll_controller.js
import { Controller } from 'stimulus'

export default class extends Controller {
  connect () {
    this.scroll = this.scroll.bind(this)
    const config = { childList: true }

    this.observer = new MutationObserver(this.scroll)
    this.observer.observe(this.element, config)
  }

  disconnect () {
    this.observer.disconnect()
  }

  scroll (mutationsList, observer) {
    for (const mutation of mutationsList) {
      if (mutation.type === 'childList') {
        const children = this.element.children
        children[children.length - 1].scrollIntoView()
      }
    }
  }
}

This will scroll to the end even if you're trying to view higher up the logs. Might look at trying to handle that. Otherwise it works how I'd expect.

Colors

The cloudsh application outputs colors to the console using ANSI escape codes. I used some code from this ANSI escape code with html tags in Ruby? StackOverflow answer. Probably need something more robust, but it works for this proof of concept.