From: Daniel Kiper <hidden> Date: 2013-01-04 17:02:14
On Fri, Jan 04, 2013 at 02:38:44PM +0000, David Vrabel wrote:
On 04/01/13 14:22, Daniel Kiper wrote:
quoted
On Wed, Jan 02, 2013 at 11:26:43AM +0000, Andrew Cooper wrote:
quoted
On 27/12/12 18:02, Eric W. Biederman wrote:
quoted
Andrew Cooper[off-list ref] writes:
quoted
On 27/12/2012 07:53, Eric W. Biederman wrote:
quoted
The syscall ABI still has the wrong semantics.
Aka totally unmaintainable and umergeable.
The concept of domU support is also strange. What does domU support even mean, when the dom0 support is loading a kernel to pick up Xen when Xen falls over.
There are two requirements pulling at this patch series, but I agree
that we need to clarify them.
It probably make sense to split them apart a little even.
Thinking about this split, there might be a way to simply it even more.
/sbin/kexec can load the "Xen" crash kernel itself by issuing
hypercalls using /dev/xen/privcmd. This would remove the need for
the dom0 kernel to distinguish between loading a crash kernel for
itself and loading a kernel for Xen.
Or is this just a silly idea complicating the matter?
This is impossible with current Xen kexec/kdump interface.
It should be changed to do that. However, I suppose that
Xen community would not be interested in such changes.
I don't see why the hypercall ABI cannot be extended with new sub-ops
that do the right thing -- the existing ABI is a bit weird.
I plan to start prototyping something shortly (hopefully next week) for
the Xen kexec case.
Wow... As I can this time Xen community is interested in...
That is great. I agree that current kexec interface is not ideal.
David, I am happy to help in that process. However, if you wish I could
carry it myself. Anyway, it looks that I should hold on with my
Linux kexec/kdump patches.
My .5 cents:
- We should focus on KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload;
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2;
load should __LOAD__ kernel image and other things into hypervisor memory;
I suppose that allmost all things could be copied from linux/kernel/kexec.c,
linux/arch/x86/kernel/{machine_kexec_$(BITS).c,relocate_kernel_$(BITS).c};
I think that KEXEC_CMD_kexec should stay as is,
- Hmmm... Now I think that we should still use kexec syscall to load image
into Xen memory (with new KEXEC_CMD_kexec_load2) because it establishes
all things which are needed to call kdump if dom0 crashes; however,
I could be wrong...
- last but not least, we should think about support for PV guests too.
Daniel
From: David Vrabel <hidden> Date: 2013-01-10 14:20:00
On 04/01/13 17:01, Daniel Kiper wrote:
On Fri, Jan 04, 2013 at 02:38:44PM +0000, David Vrabel wrote:
quoted
On 04/01/13 14:22, Daniel Kiper wrote:
quoted
On Wed, Jan 02, 2013 at 11:26:43AM +0000, Andrew Cooper wrote:
quoted
On 27/12/12 18:02, Eric W. Biederman wrote:
quoted
Andrew Cooper[off-list ref] writes:
quoted
On 27/12/2012 07:53, Eric W. Biederman wrote:
quoted
The syscall ABI still has the wrong semantics.
Aka totally unmaintainable and umergeable.
The concept of domU support is also strange. What does domU support even mean, when the dom0 support is loading a kernel to pick up Xen when Xen falls over.
There are two requirements pulling at this patch series, but I agree
that we need to clarify them.
It probably make sense to split them apart a little even.
Thinking about this split, there might be a way to simply it even more.
/sbin/kexec can load the "Xen" crash kernel itself by issuing
hypercalls using /dev/xen/privcmd. This would remove the need for
the dom0 kernel to distinguish between loading a crash kernel for
itself and loading a kernel for Xen.
Or is this just a silly idea complicating the matter?
This is impossible with current Xen kexec/kdump interface.
It should be changed to do that. However, I suppose that
Xen community would not be interested in such changes.
I don't see why the hypercall ABI cannot be extended with new sub-ops
that do the right thing -- the existing ABI is a bit weird.
I plan to start prototyping something shortly (hopefully next week) for
the Xen kexec case.
Wow... As I can this time Xen community is interested in...
That is great. I agree that current kexec interface is not ideal.
I spent some more time looking at the existing interface and
implementation and it really is broken.
David, I am happy to help in that process. However, if you wish I could
carry it myself. Anyway, it looks that I should hold on with my
Linux kexec/kdump patches.
I should be able to post some prototype patches for Xen in a few weeks.
No guarantees though.
My .5 cents:
- We should focus on KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload;
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2;
load should __LOAD__ kernel image and other things into hypervisor memory;
Yes, but I don't see how we can easily support both ABIs easily. I'd be
in favour of replacing the existing hypercalls and requiring updated
kexec tools in dom0 (this isn't that different to requiring the correct
libxc in dom0).
I suppose that allmost all things could be copied from linux/kernel/kexec.c,
linux/arch/x86/kernel/{machine_kexec_$(BITS).c,relocate_kernel_$(BITS).c};
I think that KEXEC_CMD_kexec should stay as is,
I don't think we want all the junk from Linux inside Xen -- we only want
to support the kdump case and do not have to handle returning from the
kexec image.
- Hmmm... Now I think that we should still use kexec syscall to load image
into Xen memory (with new KEXEC_CMD_kexec_load2) because it establishes
all things which are needed to call kdump if dom0 crashes; however,
I could be wrong...
I don't think we need the kexec syscall. The kernel can unconditionally
do the crash hypercall, which will return if the kdump kernel isn't
loaded and the kernel can fall back to the regular non-kexec panic.
This will allow the kexec syscall to be used only for the domU kexec case.
- last but not least, we should think about support for PV guests
too.
I won't be looking at this.
To avoid confusion about the two largely orthogonal sorts of kexec how
about defining some terms. I suggest:
Xen kexec: Xen executes the image in response to a Xen crash or a
hypercall from a privileged domain.
Guest kexec: The guest kernel executes the images within the domain in
response to a guest kernel crash or a system call.
David
From: Daniel Kiper <hidden> Date: 2013-01-11 13:22:57
On Thu, Jan 10, 2013 at 02:19:55PM +0000, David Vrabel wrote:
On 04/01/13 17:01, Daniel Kiper wrote:
quoted
On Fri, Jan 04, 2013 at 02:38:44PM +0000, David Vrabel wrote:
quoted
On 04/01/13 14:22, Daniel Kiper wrote:
quoted
On Wed, Jan 02, 2013 at 11:26:43AM +0000, Andrew Cooper wrote:
quoted
On 27/12/12 18:02, Eric W. Biederman wrote:
quoted
Andrew Cooper[off-list ref] writes:
quoted
On 27/12/2012 07:53, Eric W. Biederman wrote:
quoted
The syscall ABI still has the wrong semantics.
Aka totally unmaintainable and umergeable.
The concept of domU support is also strange. What does domU support even mean, when the dom0 support is loading a kernel to pick up Xen when Xen falls over.
There are two requirements pulling at this patch series, but I agree
that we need to clarify them.
It probably make sense to split them apart a little even.
Thinking about this split, there might be a way to simply it even more.
/sbin/kexec can load the "Xen" crash kernel itself by issuing
hypercalls using /dev/xen/privcmd. This would remove the need for
the dom0 kernel to distinguish between loading a crash kernel for
itself and loading a kernel for Xen.
Or is this just a silly idea complicating the matter?
This is impossible with current Xen kexec/kdump interface.
It should be changed to do that. However, I suppose that
Xen community would not be interested in such changes.
I don't see why the hypercall ABI cannot be extended with new sub-ops
that do the right thing -- the existing ABI is a bit weird.
I plan to start prototyping something shortly (hopefully next week) for
the Xen kexec case.
Wow... As I can this time Xen community is interested in...
That is great. I agree that current kexec interface is not ideal.
I spent some more time looking at the existing interface and
implementation and it really is broken.
quoted
David, I am happy to help in that process. However, if you wish I could
carry it myself. Anyway, it looks that I should hold on with my
Linux kexec/kdump patches.
I should be able to post some prototype patches for Xen in a few weeks.
No guarantees though.
That is great. If you need any help drop me a line.
quoted
My .5 cents:
- We should focus on KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload;
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2;
load should __LOAD__ kernel image and other things into hypervisor memory;
Yes, but I don't see how we can easily support both ABIs easily. I'd be
in favour of replacing the existing hypercalls and requiring updated
kexec tools in dom0 (this isn't that different to requiring the correct
libxc in dom0).
Why? Just define new strutures for new functions of kexec hypercall.
That should suffice.
quoted
I suppose that allmost all things could be copied from linux/kernel/kexec.c,
linux/arch/x86/kernel/{machine_kexec_$(BITS).c,relocate_kernel_$(BITS).c};
I think that KEXEC_CMD_kexec should stay as is,
I don't think we want all the junk from Linux inside Xen -- we only want
to support the kdump case and do not have to handle returning from the
kexec image.
I do not want to implement kexec jump or stuff like. However, I think that
it is worth use code which could be used. As I know there are lot of stuff
which was taken with smaller or bigger changes from Linux Kernel.
Why we would like to reinvent the wheel this time?
Additionally, we should not drop kexec support. It is main part of kdump.
In case of kdump new kernel (and other stuff) is placed in prealocated
space in contrary to kexec. That's all. kexec is useful if you would like
to quickly (skipping BIOS) switch from Xen to baremetal Linux. If you drop
kexec support from Xen then you need alter kexec-tools package in bunch
of distros to take into account new Xen behavior.
I think that it is not we want to do.
quoted
- Hmmm... Now I think that we should still use kexec syscall to load image
into Xen memory (with new KEXEC_CMD_kexec_load2) because it establishes
all things which are needed to call kdump if dom0 crashes; however,
I could be wrong...
I don't think we need the kexec syscall. The kernel can unconditionally
do the crash hypercall, which will return if the kdump kernel isn't
loaded and the kernel can fall back to the regular non-kexec panic.
No, please do not do that. When you call HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
system is completly shutdown. Return form HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
would require to restore some kernel functionalities. It maybe impossible
in some cases. Additionally, it means that some changes should be made
in generic kexec code path. As I know kexec maintainers are very reluctant
to make such things.
This will allow the kexec syscall to be used only for the domU kexec case.
quoted
- last but not least, we should think about support for PV guests
too.
I won't be looking at this.
OK.
To avoid confusion about the two largely orthogonal sorts of kexec how
about defining some terms. I suggest:
Xen kexec: Xen executes the image in response to a Xen crash or a
hypercall from a privileged domain.
Guest kexec: The guest kernel executes the images within the domain in
response to a guest kernel crash or a system call.
From: David Vrabel <hidden> Date: 2013-01-11 15:22:40
On 11/01/13 13:22, Daniel Kiper wrote:
On Thu, Jan 10, 2013 at 02:19:55PM +0000, David Vrabel wrote:
quoted
On 04/01/13 17:01, Daniel Kiper wrote:
quoted
My .5 cents:
- We should focus on KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload;
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2;
load should __LOAD__ kernel image and other things into hypervisor memory;
Yes, but I don't see how we can easily support both ABIs easily. I'd be
in favour of replacing the existing hypercalls and requiring updated
kexec tools in dom0 (this isn't that different to requiring the correct
libxc in dom0).
Why? Just define new strutures for new functions of kexec hypercall.
That should suffice.
The current hypervisor ABI depends on an internal kernel ABI (i.e., the
ABI provided by relocate_kernel). We do not want hypervisor internals
to be constrained by having to be compatible with kernel internals.
quoted
quoted
- Hmmm... Now I think that we should still use kexec syscall to load image
into Xen memory (with new KEXEC_CMD_kexec_load2) because it establishes
all things which are needed to call kdump if dom0 crashes; however,
I could be wrong...
I don't think we need the kexec syscall. The kernel can unconditionally
do the crash hypercall, which will return if the kdump kernel isn't
loaded and the kernel can fall back to the regular non-kexec panic.
No, please do not do that. When you call HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
system is completly shutdown. Return form HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
would require to restore some kernel functionalities. It maybe impossible
in some cases. Additionally, it means that some changes should be made
in generic kexec code path. As I know kexec maintainers are very reluctant
to make such things.
Huh? There only needs to be a call to a new hypervisor_crash_kexec()
function (which would then call the Xen specific crash hypercall) at the
very beginning of crash_kexec(). If this returns the normal
crash/shutdown path is done (which could even include a guest kexec!).
David
From: Daniel Kiper <hidden> Date: 2013-01-11 17:36:34
On Fri, Jan 11, 2013 at 03:22:35PM +0000, David Vrabel wrote:
On 11/01/13 13:22, Daniel Kiper wrote:
quoted
On Thu, Jan 10, 2013 at 02:19:55PM +0000, David Vrabel wrote:
quoted
On 04/01/13 17:01, Daniel Kiper wrote:
quoted
My .5 cents:
- We should focus on KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload;
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2;
load should __LOAD__ kernel image and other things into hypervisor memory;
Yes, but I don't see how we can easily support both ABIs easily. I'd be
in favour of replacing the existing hypercalls and requiring updated
kexec tools in dom0 (this isn't that different to requiring the correct
libxc in dom0).
Why? Just define new strutures for new functions of kexec hypercall.
That should suffice.
The current hypervisor ABI depends on an internal kernel ABI (i.e., the
ABI provided by relocate_kernel). We do not want hypervisor internals
to be constrained by having to be compatible with kernel internals.
I agree. I did not sugest to stay with current interface. Old KEXEC_CMD_kexec_load
and KEXEC_CMD_kexec_unload should stay as is for backward compatibility (maybe
someday they should be removed). However, I do not see any problem in adding
new KEXEC_CMD_kexec_load2 and KEXEC_CMD_kexec_unload2 functions with completely
new arguments to existing kexec hypercall. Let's say something like that:
struct kexec_segment {
void *buf;
size_t bufsz;
unsigned long mem;
size_t memsz;
};
struct xen_kexec_load2 {
unsigned long entry;
unsigned long nr_segments;
struct kexec_segment *segments;
unsigned long flags;
};
struct xen_kexec_load2 xkl2;
...
rc = HYPERVISOR_kexec_op(KEXEC_CMD_kexec_load2, &xkl2);
Regarding relocate_kernel(), it should be Xen hypervisor specific but
probably most of the code will be similar to its Linux Kernel version.
It should only at the end leave machine in state identical with state
left by Linux Kernel version of relocate_kernel(). Just to be compatible
with existing kexec/kdump implementations.
quoted
quoted
quoted
probably we should introduce KEXEC_CMD_kexec_load2 and KEXEC_CMD_k
quoted
quoted
quoted
- Hmmm... Now I think that we should still use kexec syscall to load image
into Xen memory (with new KEXEC_CMD_kexec_load2) because it establishes
all things which are needed to call kdump if dom0 crashes; however,
I could be wrong...
I don't think we need the kexec syscall. The kernel can unconditionally
do the crash hypercall, which will return if the kdump kernel isn't
loaded and the kernel can fall back to the regular non-kexec panic.
No, please do not do that. When you call HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
system is completly shutdown. Return form HYPERVISOR_kexec_op(KEXEC_CMD_kexec)
would require to restore some kernel functionalities. It maybe impossible
in some cases. Additionally, it means that some changes should be made
in generic kexec code path. As I know kexec maintainers are very reluctant
to make such things.
Huh? There only needs to be a call to a new hypervisor_crash_kexec()
function (which would then call the Xen specific crash hypercall) at the
very beginning of crash_kexec(). If this returns the normal
crash/shutdown path is done (which could even include a guest kexec!).
I am still not convinced. Howerver, go ahead with your vision in this case.
Later we will see it makes sense.
Daniel