Tracking Rails Main
Running low on new features to implement, I took to working on a rails upgrade for On Guard Security Training.
Not just any upgrade, but I set out to track rails main.
What exactly is rails main, and what is required to use it?
Rails main is the master branch of the rails framework, and some very large organizations are opting into riding the bleeding edge of rails, much more so recently than in years past. Github found themselves on a custom fork of rails 3, and @eileencodes talks frequently about this project. While not as big as Github, the custom fork of Canvas has complexity beyond what I’m even familiar.
The process begins with a simple gemfile update.
gem 'rails', '6.0.4'
We are going from
6 leveraging the work
already in place (thus the specific version). Once
we get from 5.2 to 6, we will go through 6.1, then
With the gemfile updated, running bundle update will pick up the latest version of all the dependencies
I received the following on my mac trying to update:
Could not find MIME type database in the following locations: ["/usr/local/share/mime/packages/freedesktop.org.xml", "/opt/homebrew/share/mime/packages/freedesktop.org.xml", "/opt/local/share/mime/packages/freedesktop.org.xml", "/usr/share/mime/packages/freedesktop.org.xml"] An error occurred while installing mimemagic (0.3.10), and Bundler cannot continue. Make sure that `gem install mimemagic -v '0.3.10' --source 'https://rubygems.org/'` succeeds before bundling.
Locally, this was resolved with
brew install shared-mime-info
and the servers took
sudo apt-get isntall shared-mime-info
Running the new upgrade script is dangerous. It will
attempt to overwrite some very important files. I don’t
have good advice for this, because I just said no to
.rb files like
than losing hundreds of lines of configuration.
After booting my app, I got a stack trace that is indecipherable so to make things worse, I decided to push towards ruby-2.7.0, which will be required to make main work.
rvm install 2.7 rvm use 2.7 bundle install
Same stack trace, good enough time to dig in. Found this
in the stack trace right before a Runtime Error:
Checked current rack version, and updated gemfile to 2.2.3.
Finally got a page load, but is still a 500 error:
undefined method 'delete' for "8201ea794c9d6a6e2215ec98938b1642":ActionDispatch::Session::CookieStore::SessionId
Tried the request again in an incognito browser and
same results. Checking further in the stacktrace,
it was clear middleware injections were causing
the crash, before the app was even loaded. A change
to migration context (
takes an extra parameter.
After fixing this, another dependency was throwing
the same 1 argument given expected 2, and updating
the dependency allowed the main page to load. The only
additional thing required to make the main page load
was replacing a
render 'html' with
Moving to 6.1
With ruby 2.7 and rails 6.0.4, the next step is to
replace rails 6.0.4 with 6.1.4 and
switchman (= 1.14.7) was resolved to 1.14.7, which depends on shackles (~> 1.4.2) was resolved to 1.4.2, which depends on railties (>= 4.0, < 6.1) switchman (= 1.14.7) was resolved to 1.14.7, which depends on railties (>= 5.0, < 6.1) switchman-inst-jobs (= 1.3.7) was resolved to 1.3.7, which depends on railties (>= 4.2, < 6.1)
These are very concerning results, because these gems
are produced by the same company who open sourced
the learning platform being used.
Switchman for example
provided the database sharding that didn’t exist in
core until 6.1, and there hasn’t been good effort in these
gems to move towards 6.1 compatibility. We aren’t using
sharding, so a potential solution is to remove all the
code that allows sharding.
shackles has been replaced with
guardrails which leans into the 6.1 way of doing things.
Eventually, the dependencies were found in a way that would work without having to manually edit the dependency chain (which will likely be done to get main support) However, the entire process ended with:
Unfortunately, an unexpected error occurred, and Bundler cannot continue.
I’ve been running on
bundler 1.17, so maybe its time to
update that. A quickstart for bundler 2 is available.
bundle update --bundler. This didn’t do
anything for me, so I updated my Gemfile bundler dependency
to allow the version of the v2 gem (2.2.21) that I had installed
and for good measure wiped the
After replacing many more (especially infrastructure) dependencies progress became stuck on
academic_benchmarks (= 1.1.1) academic_benchmark was resolved to 1.1.0, which depends on academic_benchmarks (~> 0.0)
Reading the gem README, the gem is used for consuming an
api that will never be used. Like the sqlite gem, both
are removed from the Gemfile. Canvas is a big program,
that can be used in many ways, and I’m taking my customizations
to a fuller extent.
academic_benchmark turned out to
be a local gem which could be set to allow version
academic_benchmarks. Be careful, they look similar.
Canvas also has many gems included in distribution, and
a lot of these gems use
'~>' specifications. To ask
for the potentially breaking semver changes, those are
'>=' definitions. Sometimes embedded
gemfiles are just commented out, like
is little chance of moving from aws to google for storage.
The biggest challenge is that
Shackles was replaced
GuardRail, so a naive replace all in path was
used. Then something challenging happened, a
occurred inside of a gem. The rails code calling was
t("some text") which was aliased in i18n initializer
I18n.t("some text") was
working, the initializer which overrode some behavior
was monkeypatched to catch the
pass through the string sent to translation with:
rescue I18n::ArgumentError return args.dig(0,:default,0) end
Finally, the homepage would not load the courses
GradingPeriod a more specific binding
to a gem. Luckily RubyMine does a good job
of finding the class of a given method by Command-clicking
on the method.
Canvas adopted sharding early on, somewhere around 2015
by implementing their own Gem for handling writing
to multiple tables/databases to improve performance.
As rails adopted a concept of sharding in 6.1, a lot
of work needs to go into integrating the new concept
of sharding. At present,
shard_category is missing
from presumably enrollments, which may assume the
correct mixin of Switchman is still missing in
ActiveRecord::Base. The depth of this fix may be
too much to push through in a weekend. For now,
we will default to the primary shard:
db = if klass.respond_to? :shard_category Shard.current(klass.shard_category).database_server else Shard.current.database_server end
These steps will need to be followed one final time to get
to the main branch, but with all the static dependencies
in this project, creating forks of all the gems will be
required, which will be even more work. Update the
.gemspec files and point the Gemfile to
Before proceeding to the rails main branch
git: 'https://github.com/rails/rails', more conclusive
testing will be required. It is important to ensure these
major update breaks are resolved before moving to the
next major version.
See the code on github, at andygauge/canvas-lms
Part 2: actually tracking rails main
Next we move from 6.0 to main on a smaller app before undertaking this major change.