This makes insertion to block list O(log n) instead of O(n) when
blocking with wake time. This is more heavy and currently slower than
the previous implementation, but that is just because we don't really
have too many threads running at any given time. This will be more
scalable in the future and even allows implementing priorities if I ever
wish so
Now modifying Scheduler.h or Thread.h doesnt trigger practically a full
kernel rebuild. This required moving Mutex and RWLock of of line but
that should be fine :^)
The old SpinLockAsMutex was pretty confusing first of all. It also was
causing the issue thats been around for maybe a year now which is the
only consistently happening kernel panic. I've been pretty confused
about this and finally figured out what was causing this.
The main issue was that we were accidentally enabling interrupts when
blocking a thread that passed SpinLockAsMutex from normally interrupt
enabled context. This led to receiving IPI for thread unblock while
we were actively blocking the thread. I'm very suprized this had't
caused any more serious issues than occasional kernel panics :^)
Don't wake up threads when getting the next timer deadline.
If we are idling this can: remove thread from block list, get the
deadline for the next blocked thread, return to idle thread. This will
end up sleeping until the next wake up condition which in the worst
case would be rebalance, or indefinitely when not running SMP.
Make sure we always set the next timer deadline. I was not doing that if
the timer interrupt did not lead to a yield
Scheduler will now set a timer interrupt deadline for when it should get
interrupted next. This fixes a big issue I've had for a long time where
sleeping on an idle core would have to wait for the next periodic
interrupt to wake up. This could cause 1 ms sleeps to be close to 10 ms.
This currently only supports lapic timers and will still fallback to
periodic interrupts when running with PIC. I should add deadline support
to PIT/HPET but why would you run with APIC disabled ;)
I don't know why I though the block chain had to be stored fully in the
ThreadBlocker, that did not even fix the problem I was trying to fix
when I last rewrote it. Roll back to doubly linked list of block chain
and now just check that the node is contained within the ThreadBlocker
before removing and after acquiring the ThreadBlocker's lock. Also there
is no need to have a separate lock the node's blocker field. We can just
perform an atomic reads and writes to it. We can still get a blocker
that the node is no longer part of, but this can be resolved with a
simple check. This patch reduces ThreadBlocker's size from over 200
bytes to just 12 bytes +4 bytes padding
Instead of keeping track of the current time and rescheduling when
interval has passed, keep track of the next expected reschedule time.
This prevents theoretically missing every second pre-emption when
scheduler's timer is interrupting at same rate as the interval.
There is no need to save and load sse state on every interrupt. Instead
we can use CR0.TS to make threads trigger an interrupt when they use sse
instructions. This can be used to only save and load sse state when
needed.
Processor now keeps track of its current "sse thread" and the scheduler
either enabled or disabled sse based on which thread it is starting up.
When a thread dies, it checks if it was the current sse thread to avoid
use after free bugs. When load balancing, processor has to save the
thread's sse state before sending it to a new processor (if it was the
current sse thread). This ensures thread's sse state will be correct
when the new processor ends up loading it.
Doing a yield no longer raises a software interrupt. Instead it just
saves all the callee saved registers, ip, sp and return value. Because
yield is only called in the kernel, it can just restore registers and
jump to the target address. There is never a need to use iret :)
My code to find least loaded processor used processor index instead of
processor id to index the array. Most of the time this lead to wrong
processor returned as the least loaded, leaving some processors
basically idle.
If the processor has invariant TSC it can be used to measure time. We
keep track of the last nanosecond and TSC values and offset them based
on the current TSC. This allows getting current time in userspace.
The implementation maps a single RO page to every processes' address
space. The page contains the TSC info which gets updated every 100 ms.
If the processor does not have invariant TSC, this page will not
indicate the capability for TSC based timing.
There was the problem about how does a processor know which cpu it is
running without doing syscall. TSC counters may or may not be
synchronized between cores, so we need a separate TSC info for each
processor. I ended up adding sequence of bytes 0..255 at the start of
the shared page. When a scheduler gets a new thread, it updates the
threads gs/fs segment to point to the byte corresponding to the current
cpu.
This TSC based timing is also used in kernel. With 64 bit HPET this
probably does not bring much of a benefit, but on PIT or 32 bit HPET
this removes the need to aquire a spinlock to get the current time.
This change does force the userspace to not use gs/fs themselves and
they are both now reserved. Other one is used for TLS (this can be
technically used if user does not call libc code) and the other for
the current processor index (cannot be used as kernel unconditionally
resets it after each load balance).
I was looking at how many times timer's current time was polled
(userspace and kernel combined). When idling in window manager, it was
around 8k times/s. When running doom it peaked at over 1 million times
per second when loading and settled at ~30k times/s.
Before this real hardware failed to boot with smp enabled. Allocating
the idle thread does a page mapping which ends up broadcasting TLB
shootdown to other processes. This ends up failing somewhere halting the
processors never allowing them to initialize their scheduler
Kernel can just use raw threads, pretty muchs the only thing that
process provides is syscalls which kernel threads of course don't
need.
Also this makes init process have pid 1 :D
This gets rid of a very old bug where kernel panics when thread is being
woken up and unblocked at the same time on different cores. This
required adding a new lock to SchedulerQueue::Node and adding a cap to
how many threads a threadblocker can simultaneously block. I don't think
I ever block more than five threads on the same ThreadBlocker so this
should be fine.
All block functions now take an optional mutex parameter that is
atomically unlocked instead of having the user unlock it before hand.
This prevents a ton of race conditions everywhere in the code!
Scheduler keeps crashing all the time when running on multiple cores.
This patch disabled the load balancer, which seems to get rid of most
scheduler crashes.
If thread was blocked, but had not reached block queue, you might
already get an unblock request which would fail on an assertion.
If blocked thread was load balanced to another processor and unblocked
simultaneously, there was a race condition.
This makes scheduler preemption much cleaner as bsb does not have to
send smp messages to notify other processes about timer interrupt.
Also PIT percision is now "full" 0.8 us instead of 1 ms that I was using
before.
Change Semaphore -> ThreadBlocker
This was not a semaphore, I just named it one because I didn't know
what semaphore was. I have meant to change this sooner, but it was in
no way urgent :D
Implement SMP events. Processors can now be sent SMP events through
IPIs. SMP events can be sent either to a single processor or broadcasted
to every processor.
PageTable::{map_page,map_range,unmap_page,unmap_range}() now send SMP
event to invalidate TLB caches for the changed pages.
Scheduler no longer uses a global run queue. Each processor has its own
scheduler that keeps track of the load on the processor. Once every
second schedulers do load balancing. Schedulers have no access to other
processors' schedulers, they just see approximate loads. If scheduler
decides that it has too much load, it will send a thread to another
processor through a SMP event.
Schedulers are currently run using the timer interrupt on BSB. This
should be not the case, and each processor should use its LAPIC timer
for interrupts. There is no reason to broadcast SMP event to all
processors when BSB gets timer interrupt.
Old scheduler only achieved 20% idle load on qemu. That was probably a
very inefficient implementation. This new scheduler seems to average
around 1% idle load. This is much closer to what I would expect. On my
own laptop idle load seems to be only around 0.5% on each processor.
Current context saving was very hacky and dependant on compiler
behaviour that was not consistent. Now we always use iret for
context saving. This makes everything more clean.
Bananymous