README - pub/scm/linux/kernel/git/mcgrof/linker-tables - Git at Google


 Compile
 =======

 Run:
 	make

 Simulated boots
 ==============

 This programs simulates boot in on both bare metal and xen.
 The goal of the code is to illustrate an issue of dead code
 and how we can fix this.

 Emulate bare metal boot:

 ./main

 Emulate xen boot:

 ./main -x

 Section ranges and tables
 =========================

 There are standard ELF sections and over time Linux has defined a series of
 custom ELF sections which are now de facto Linux sections. Together we refer
 to these as Linux ELF sections. History shows there are two forms in which Linux
 sections can be used for:

  o Section ranges - typically used for grouping a set of executable functions
  o Section tables - typically used for grouping a set of data structures

 In order to help build concrete and clean, separate semantics for both, we
 provide separate helpers for code to declare and use them.  There is also
 a common set of helpers which both make use of, this is provided in
 include/linux/sections.h -- the file consists of of all known Linux ELF
 sections, a few helpers used for alignment, size, and building section names.
 Two common set of helpers are used to help distinguish whether or not a
 particular section range or section table is const or not:

 #define DECLARE_LINUX_SECTION(type, name)				\
 	extern type name[], name##__end[];

 #define DECLARE_LINUX_SECTION_RO(type, name)				\
 	extern const type name[], name##__end[];

 Linux section ranges only use DECLARE_LINUX_SECTION_RO(). So the only thing
 else needed is to be able to specify the actual Linux section associated with
 the range. You use DECLARE_SECTION_RANGE() to declare section ranges with an
 associated name on header files, this lets other code access it. You define
 section ranges in code with DEFINE_SECTION_RANGE() which associates the
 specific section range with a Linux section.

 Linux section tables can be of const or non-const type as such can use either
 DECLARE_LINUX_SECTION() or DECLARE_LINUX_SECTION_RO() under the hood. Helpers
 are provided to help associate Linux section tables to specific sections with
 an appropriate common helper declaration.

 The two main section types, section ranges, section tables, then are referred
 to with the following macros for declaring each on their respective header
 files so others can refer to them; and defining them in the respective code
 that defines its proper use:

 -------------------------------------------------------------------------------
    Section type     |ELF name|    Declared on header   |    Defined in code
 -------------------------------------------------------------------------------
 SECTION_TYPE_RANGES |.ranges | DECLARE_SECTION_RANGE() | DEFINE_SECTION_RANGE()
 SECTION_TYPE_TABLES |.ranges |  DECLARE_LINKTABLE*()   | DEFINE_LINKTABLE*()
 -------------------------------------------------------------------------------

 There are only two declarations for linker tables, which would be used in
 header files:

   o DECLARE_LINKTABLE()		for data
   o DECLARE_LINKTABLE_RO()	for read-only data

 The tables have a series of definitions (DEFINE_LINKTABLE_*()) given to help
 with setup for the different supported Linux sections used. The definitions
 would be used in the code defining the use of tables -- the actual
 implementation.

 The definitions currently supported for tables are:

 -------------------------------------------------------------------------------
 Linux section    |    const-or-not  |  Definition
 -------------------------------------------------------------------------------
 .data            |    non-const     |  DEFINE_LINKTABLE()
 .text            |    const         |  DEFINE_LINKTABLE_TEXT()
 .init            |    const         |  DEFINE_LINKTABLE_INIT()
 .init.data       |    non-const     |  DEFINE_LINKTABLE_INIT_DATA()
 .rodata          |    const         |  DEFINE_LINKTABLE_RO()
 -------------------------------------------------------------------------------

 x86 bzimage parser
 ================

 An x86 image parser has been added to aid the integration requirements
 of using futher boot_params for dead code concerns. Example usage:

 ./parse-bzimage ~/linux/arch/x86/boot/bzImage
 kernel: /home/mcgrof/linux/arch/x86/boot/bzImage
 kernel size:    5668000 bytes
 Going to parse kernel...

 Xen Expects:    0x53726448
 Qemu Expects:   0x53726448
 On image:       0x53726448


 bzImage protocol Version: v2.13
 Xen hdr->version:       525
 Qemu protocol:          525
 Qemu VERSION(2,8):      520

 -------------------------------------------------
 Boot protocol 2.07:     0x0207  (supports hardware_subarch)
 Boot protocol 2.08:     0x0208
 Boot protocol 2.09:     0x0209
 Boot protocol 2.10:     0x020a
 Boot protocol 2.11:     0x020b
 Boot protocol 2.12:     0x020c
 Boot protocol 2.13:     0x020d


 Member                                  Offset  Expected        Match
 -------------------------------------------------------------------------
 setup_header->loadflags                 0x0211  0x0211          YES
 setup_header->hardware_subarch          0x023c
 setup_header->hardware_subarch_data     0x0240

 TODO
 ====

  o Finish checking LTO_REFERENCE_INITCALL() is indeed needed. I've aksed [0].
    so now just waiting on that discussion to complete. If its not needed,
    rip it out of the Linux kernel and then from here.

 [0] https://lkml.kernel.org/r/CAB=NE6W6Eh1tU=NXf20f-CPYeT22mtXGzrYqM9W-f14dGBrEsA@mail.gmail.com

 Table solution history
 ======================

 The init table solution implemented here is a combination of how the
 Linux IOMMU init framework was implemented *and* how GPXE sorts its
 own init work.

 Table sorting
 =============

 The Linux IOMMU solution gives us dependency annotations,
 so one set of routines will not run until its dependency runs. The
 sorting for this takes place with a C routine which ultimately
 memmove()'s the routines in order.

 The sorting done by the GPXE solution is all done by the linker
 by the good 'ol SORT() through in the object's linker script.

 We want to enable different subsystems to implement their own
 table solutions and this can mean having different init structure
 members, that is how the gpxe solution works. Sorting is done automatically
 for you through linker at build time. The sorting through the IOMMU init
 solution works using the actual specific structure for the subsystem
 used and the semantics for the sorting must be clearly documented and
 defined. For instance the original IOMMU init solution requires only
 one init solution to have a NULL set for its depends callback. For
 our mockup x86 init solution we are allowing both "depends" and
 "detect" to be NULL. For changes to how our sorting implementation
 was modified refer to commit 9e01858bbc74a11 ("init: enable support
 for no depends() callback). The sorting for routines that do not have
 a "depends" callback set would happen through the order table level
 used, a la gpxe solution. When the order levels are the same we rely
 on order of linking -- for instance: placement of the routines in the
 C file, and at the file level based on order in the Makefile.

 Furthermore, upon initialization the mockup x86 init solution relies
 on the subarchitecture to skip routines which are not supported or
 intended for its subarchitecture.

 It might be possible to share a common C sort routine for all tables
 but the first challenge is we'd likely be constraining our init structures.
 For instance since the current sorting looks for an init routine's "depends"
 callback, that callback will be placed in a specific location as part of the
 structure. In order to share a sort we'd have to enforce a common set of
 table structures for at least the sorting semantics, for instance this
 could perhaps be accomplished by placing all relevant members relevant to
 sorting semantics in the beginning of the structure. The next challenge
 has to do with size, given that we move structures around with memmove()
 and this is done depending on the size of the init structure, and that
 is up to each subsystem to define. This could be resolved by fixating the
 size of all table init structures.

 For now each subsystem must then define its own sorting scheme. A sorting
 algorithm is only necessary if the linker script sorting does not suffice
 for the semantics of the subsystem.

 File mapping upstream
 =====================

 In case this project maintains a matching with upstream this project strives
 to maintain a one to one direct mapping as things are or we hope would
 be on upstream Linux.

	Compile
	=======

	Run:
	make

	Simulated boots
	==============

	This programs simulates boot in on both bare metal and xen.
	The goal of the code is to illustrate an issue of dead code
	and how we can fix this.

	Emulate bare metal boot:

	./main

	Emulate xen boot:

	./main -x

	Section ranges and tables
	=========================

	There are standard ELF sections and over time Linux has defined a series of
	custom ELF sections which are now de facto Linux sections. Together we refer
	to these as Linux ELF sections. History shows there are two forms in which Linux
	sections can be used for:

	o Section ranges - typically used for grouping a set of executable functions
	o Section tables - typically used for grouping a set of data structures

	In order to help build concrete and clean, separate semantics for both, we
	provide separate helpers for code to declare and use them. There is also
	a common set of helpers which both make use of, this is provided in
	include/linux/sections.h -- the file consists of of all known Linux ELF
	sections, a few helpers used for alignment, size, and building section names.
	Two common set of helpers are used to help distinguish whether or not a
	particular section range or section table is const or not:

	#define DECLARE_LINUX_SECTION(type, name) \
	extern type name[], name##__end[];

	#define DECLARE_LINUX_SECTION_RO(type, name) \
	extern const type name[], name##__end[];

	Linux section ranges only use DECLARE_LINUX_SECTION_RO(). So the only thing
	else needed is to be able to specify the actual Linux section associated with
	the range. You use DECLARE_SECTION_RANGE() to declare section ranges with an
	associated name on header files, this lets other code access it. You define
	section ranges in code with DEFINE_SECTION_RANGE() which associates the
	specific section range with a Linux section.

	Linux section tables can be of const or non-const type as such can use either
	DECLARE_LINUX_SECTION() or DECLARE_LINUX_SECTION_RO() under the hood. Helpers
	are provided to help associate Linux section tables to specific sections with
	an appropriate common helper declaration.

	The two main section types, section ranges, section tables, then are referred
	to with the following macros for declaring each on their respective header
	files so others can refer to them; and defining them in the respective code
	that defines its proper use:

	-------------------------------------------------------------------------------
	Section type \|ELF name\| Declared on header \| Defined in code
	-------------------------------------------------------------------------------
	SECTION_TYPE_RANGES \|.ranges \| DECLARE_SECTION_RANGE() \| DEFINE_SECTION_RANGE()
	SECTION_TYPE_TABLES \|.ranges \| DECLARE_LINKTABLE() \| DEFINE_LINKTABLE()
	-------------------------------------------------------------------------------

	There are only two declarations for linker tables, which would be used in
	header files:

	o DECLARE_LINKTABLE() for data
	o DECLARE_LINKTABLE_RO() for read-only data

	The tables have a series of definitions (DEFINE_LINKTABLE_*()) given to help
	with setup for the different supported Linux sections used. The definitions
	would be used in the code defining the use of tables -- the actual
	implementation.

	The definitions currently supported for tables are:

	-------------------------------------------------------------------------------
	Linux section \| const-or-not \| Definition
	-------------------------------------------------------------------------------
	.data \| non-const \| DEFINE_LINKTABLE()
	.text \| const \| DEFINE_LINKTABLE_TEXT()
	.init \| const \| DEFINE_LINKTABLE_INIT()
	.init.data \| non-const \| DEFINE_LINKTABLE_INIT_DATA()
	.rodata \| const \| DEFINE_LINKTABLE_RO()
	-------------------------------------------------------------------------------

	x86 bzimage parser
	================

	An x86 image parser has been added to aid the integration requirements
	of using futher boot_params for dead code concerns. Example usage:

	./parse-bzimage ~/linux/arch/x86/boot/bzImage
	kernel: /home/mcgrof/linux/arch/x86/boot/bzImage
	kernel size: 5668000 bytes
	Going to parse kernel...

	Xen Expects: 0x53726448
	Qemu Expects: 0x53726448
	On image: 0x53726448


	bzImage protocol Version: v2.13
	Xen hdr->version: 525
	Qemu protocol: 525
	Qemu VERSION(2,8): 520

	-------------------------------------------------
	Boot protocol 2.07: 0x0207 (supports hardware_subarch)
	Boot protocol 2.08: 0x0208
	Boot protocol 2.09: 0x0209
	Boot protocol 2.10: 0x020a
	Boot protocol 2.11: 0x020b
	Boot protocol 2.12: 0x020c
	Boot protocol 2.13: 0x020d


	Member Offset Expected Match
	-------------------------------------------------------------------------
	setup_header->loadflags 0x0211 0x0211 YES
	setup_header->hardware_subarch 0x023c
	setup_header->hardware_subarch_data 0x0240

	TODO
	====

	o Finish checking LTO_REFERENCE_INITCALL() is indeed needed. I've aksed [0].
	so now just waiting on that discussion to complete. If its not needed,
	rip it out of the Linux kernel and then from here.

	[0] https://lkml.kernel.org/r/CAB=NE6W6Eh1tU=NXf20f-CPYeT22mtXGzrYqM9W-f14dGBrEsA@mail.gmail.com

	Table solution history
	======================

	The init table solution implemented here is a combination of how the
	Linux IOMMU init framework was implemented and how GPXE sorts its
	own init work.

	Table sorting
	=============

	The Linux IOMMU solution gives us dependency annotations,
	so one set of routines will not run until its dependency runs. The
	sorting for this takes place with a C routine which ultimately
	memmove()'s the routines in order.

	The sorting done by the GPXE solution is all done by the linker
	by the good 'ol SORT() through in the object's linker script.

	We want to enable different subsystems to implement their own
	table solutions and this can mean having different init structure
	members, that is how the gpxe solution works. Sorting is done automatically
	for you through linker at build time. The sorting through the IOMMU init
	solution works using the actual specific structure for the subsystem
	used and the semantics for the sorting must be clearly documented and
	defined. For instance the original IOMMU init solution requires only
	one init solution to have a NULL set for its depends callback. For
	our mockup x86 init solution we are allowing both "depends" and
	"detect" to be NULL. For changes to how our sorting implementation
	was modified refer to commit 9e01858bbc74a11 ("init: enable support
	for no depends() callback). The sorting for routines that do not have
	a "depends" callback set would happen through the order table level
	used, a la gpxe solution. When the order levels are the same we rely
	on order of linking -- for instance: placement of the routines in the
	C file, and at the file level based on order in the Makefile.

	Furthermore, upon initialization the mockup x86 init solution relies
	on the subarchitecture to skip routines which are not supported or
	intended for its subarchitecture.

	It might be possible to share a common C sort routine for all tables
	but the first challenge is we'd likely be constraining our init structures.
	For instance since the current sorting looks for an init routine's "depends"
	callback, that callback will be placed in a specific location as part of the
	structure. In order to share a sort we'd have to enforce a common set of
	table structures for at least the sorting semantics, for instance this
	could perhaps be accomplished by placing all relevant members relevant to
	sorting semantics in the beginning of the structure. The next challenge
	has to do with size, given that we move structures around with memmove()
	and this is done depending on the size of the init structure, and that
	is up to each subsystem to define. This could be resolved by fixating the
	size of all table init structures.

	For now each subsystem must then define its own sorting scheme. A sorting
	algorithm is only necessary if the linker script sorting does not suffice
	for the semantics of the subsystem.

	File mapping upstream
	=====================

	In case this project maintains a matching with upstream this project strives
	to maintain a one to one direct mapping as things are or we hope would
	be on upstream Linux.