Banos is a stable WIP C driver API that is supposed to provide a simple
interface to interact with the kernel and load the modules dynamically.
It is WIP and atm this just implements module loading with a custom
banos_install syscall. Banos will not try to substitute parts of the
kernel instead it will just expose kernel functionality via a stable
BINARY API. Meaning binaries (should) remain forward and backward
compatible on a binary level.
Banos modules work similarly to those in linux, you expose symbols via
BANOS_EXPORT which allows you to export a name + addr paired symbol.
It puts it in the .banos-export section. Drivers provide metadata about
themselves in the REQUIRED .banos-driver section. Symbols are resolved
at runtime. The kernel exposes the driver functionality via the same
.banos-export export mechanism.
Banos modules are elf RELOCATABLE files (object files) which have
partial linking (only banos symbols should remain). Modules will
eventually define dependencies, will export symbols and will allow you
to build a complex object hierarchy.
This patch adds the banos_install syscall which takes in the driver
image to install and may only be executed by super users. The API
doesn't validate already loaded modules, as thats something the
userspace MAY choose to keep track of. Multi-instance functionality
shall be implemented via driver specific behaviuor (exposed in the dev
filesystem or some other means).
Modules are supposed to allow you to alter kernel behavior and extend
it, allowing you to create filesystems, drivers, networking
modifications, schedulers, probers, and more (hopefully) whilst
remaining binary compatible with any version of the kernel (again,
hopefully).
Initial step of paging now just prepares fast page for heap, actual page
table initialization happens after heap is initialized which allows
x86_64 to never depend on kmalloc for pages.
Processor's stacks are now also spawned with PMM/VMM allocated stacks
instead of kmalloc identity mapped.
Initially allocate all physical memory except kernel memory and boot
modules. Before we just skipped all memory before kernel boot modules.
Also release memory used by boot modules after the kernel is up and
running. Once the boot modules are loaded, there is no need to keep them
in memory.
If the processor has invariant TSC it can be used to measure time. We
keep track of the last nanosecond and TSC values and offset them based
on the current TSC. This allows getting current time in userspace.
The implementation maps a single RO page to every processes' address
space. The page contains the TSC info which gets updated every 100 ms.
If the processor does not have invariant TSC, this page will not
indicate the capability for TSC based timing.
There was the problem about how does a processor know which cpu it is
running without doing syscall. TSC counters may or may not be
synchronized between cores, so we need a separate TSC info for each
processor. I ended up adding sequence of bytes 0..255 at the start of
the shared page. When a scheduler gets a new thread, it updates the
threads gs/fs segment to point to the byte corresponding to the current
cpu.
This TSC based timing is also used in kernel. With 64 bit HPET this
probably does not bring much of a benefit, but on PIT or 32 bit HPET
this removes the need to aquire a spinlock to get the current time.
This change does force the userspace to not use gs/fs themselves and
they are both now reserved. Other one is used for TLS (this can be
technically used if user does not call libc code) and the other for
the current processor index (cannot be used as kernel unconditionally
resets it after each load balance).
I was looking at how many times timer's current time was polled
(userspace and kernel combined). When idling in window manager, it was
around 8k times/s. When running doom it peaked at over 1 million times
per second when loading and settled at ~30k times/s.
Kernel can just use raw threads, pretty muchs the only thing that
process provides is syscalls which kernel threads of course don't
need.
Also this makes init process have pid 1 :D
kernel now passes the name of default console to init process so init
knows which file to open as stdio. before /dev/tty was referencing the
system wide current terminal which was inherited from cmdline. This
doesn't work anymore as we have pseudo terminals implemented that can
chage the current terminal during runtime :D
This makes scheduler preemption much cleaner as bsb does not have to
send smp messages to notify other processes about timer interrupt.
Also PIT percision is now "full" 0.8 us instead of 1 ms that I was using
before.
Change Semaphore -> ThreadBlocker
This was not a semaphore, I just named it one because I didn't know
what semaphore was. I have meant to change this sooner, but it was in
no way urgent :D
Implement SMP events. Processors can now be sent SMP events through
IPIs. SMP events can be sent either to a single processor or broadcasted
to every processor.
PageTable::{map_page,map_range,unmap_page,unmap_range}() now send SMP
event to invalidate TLB caches for the changed pages.
Scheduler no longer uses a global run queue. Each processor has its own
scheduler that keeps track of the load on the processor. Once every
second schedulers do load balancing. Schedulers have no access to other
processors' schedulers, they just see approximate loads. If scheduler
decides that it has too much load, it will send a thread to another
processor through a SMP event.
Schedulers are currently run using the timer interrupt on BSB. This
should be not the case, and each processor should use its LAPIC timer
for interrupts. There is no reason to broadcast SMP event to all
processors when BSB gets timer interrupt.
Old scheduler only achieved 20% idle load on qemu. That was probably a
very inefficient implementation. This new scheduler seems to average
around 1% idle load. This is much closer to what I would expect. On my
own laptop idle load seems to be only around 0.5% on each processor.
/dev/keyboard and /dev/mouse can be read for events from any attached
keyboard or mouse respectively. This makes device hot-plugging support
pretty much automatic for TTY, GUI, and whatever takes input.
This is kinda hacky, as I had disable the PS/2 initialization so that
usb keyboard gets /dev/keyboard0. I should add device hot plugging
support for TTY and GUI...
These can allocate memory that can be shared between processes using
a global key. There is currenly no safety checks meaning anyone can
map any shared memory object just by trying to map every possible key.
Only segment 0 is supported, but devices can now be accessed through
mmio.
Adding more segments would require adding argument to every PCI API so
it is left for later.
This allows us to allocate processor stacks, and other per processor
structures dynamically in runtime. Giving processor stack to
ap_trampoline feels super hacky, but it works for now.
