Thread (10 messages) 10 messages, 6 authors, 2016-05-12

anyone aware of a high availability setup that relies on fully redundant install?

From: Andrew Bradford <hidden>
Date: 2016-05-12 16:06:48

Hi Robert,

On 04/17 10:47, Robert P. J. Day wrote:
  i figure this is as good a place as any to ask ... is anyone here
aware of anyone using a linux config and install that, for the
purposes of reliability or high availability or whatever you want to
call it, relies on a second, completely independent installation of
linux on the same hard drive?

  i was having a discussion about the reliability of upgrading linux
(any combination of kernel, basic rootfs and/or proprietary apps), and
one approach that was suggested (not by me) was to have two entirely
independent installations on the same hard drive so that, when one
"upgraded" the system, the newer content was installed in the other
partition and the system rebooted to that newer content, the rationale
being that, if the upgraded system didn't work, one could revert back
to the original install.

  the history of this particular approach lies in the fact that the
original S/W running on the system in question was an embedded app
programmed right on bare metal, so the entire app was basically a
monolithic combination of boot loader, kernel and all the apps. in
that case, i can certainly see how one wanted a reliable way to revert
back to a known working system if the new S/W had flaws. but i don't
think that logic holds with a linux installation.

  the major concern seems to be a fear that, if one upgrades the
kernel even a little bit, user space apps might suddenly stop working
because of some new incompatibility introduced by that upgrade. i'm
well aware of gregkh's piece on kernel stable API nonsense:

https://www.kernel.org/doc/Documentation/stable_api_nonsense.txt

so i'm not overly worried, but folks in the discussion coming from the
embedded programming space are using their experiences from that space
to argue that a fully redundant install is the safest approach just in
case something goes wrong with an upgrade.

  not surprisingly, my position is that this is not something they
need to worry about a whole lot but, for the sake of fairness, i
thought i'd look around to see if any linux folks *are* using that
approach.
Yes, in one product I'm working on now we have the flash partitioned out
with two separate root partitions for this reason.  Doing atomic
upgrades is not easy and this is one simple and effective way to avoid
issues arising out of failed or botched upgrades.  We're more worried
about power going out in the middle of an upgrade moreso than anything
else.

A software upgrade for us wipes the not-booted partition and installs
the entire new system into the not-booted partition, then updates the
bootloader configuration to use that as the root file system.  If the
system fails to boot 3 times (watchdog induced resets) then the
bootloader will fall back to the other partition.

To retain configuration settings across upgrades, there's yet another
partition with config data on it that both rootfs partitions use.  At
boot time the system validates that the config data is in the proper
format and fixes it if it's not.

Thanks,
Andrew
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help