ULE

在 /etc/sysctl.conf 中加入：

kern.sched.preempt_thresh=224

然后用 /etc/rc.d/sysctl start 或重启系统令其生效。

系统默认的值是 80，表示只有新优先级 < 80 时才允许抢占；224 表示非空闲线程均可以进行抢占。这样做的结果是系统会产生更多的切换，从而改善响应时间（牺牲吞吐量）。对桌面系统来说，这种设置是很有用的。

Jeff提交了ULE调度器的3.0版(sched_ule.c,v 1.200，巧合？:) 这个版本对调度器本身的上锁进行了细化，从而带来了性能改善(最上面那根黄线，之前是下面那根青色的线）。

Today, Jeff Roberson has committed his version 2.0 ULE scheduler. This new version has addressed several design issues as well as several bugs.

The new scheduler has adopted a circular queue, instead of the double-queue structure which is also found in the Linux O(1) scheduler. The latter has lead to difficulty implementing nice correctly.

For uniprocessor case, ULE is now faster.

MP algorithm has been simplified a bit.

A lot of bugfixes, etc.

To quote the original commit message:

ULE 2.0:

• Remove the double queue mechanism for timeshare threads. It was slow due to excess cache lines in play, caused suboptimal scheduling behavior with niced and other non-interactive processes, complicated priority lending, etc.
• Use a circular queue with a floating starting index for timeshare threads. Enforces fairness by moving the insertion point closer to threads with worse priorities over time.
• Give interactive timeshare threads real-time user-space priorities and place them on the realtime/ithd queue.
• Select non-interactive timeshare thread priorities based on their cpu utilization over the last 10 seconds combined with the nice value. This gives us more sane priorities and behavior in a loaded system as compared to the old method of using the interactivity score. The interactive score quickly hit a ceiling if threads were non-interactive and penalized new hog threads.
• Use one slice size for all threads. The slice is not currently dynamically set to adjust scheduling behavior of different threads.
• Add some new sysctls for scheduling parameters.

Bug fixes/Clean up:

• Fix zeroing of td_sched after initialization in sched_fork_thread() caused by recent ksegrp removal.
• Fix KSE interactivity issues related to frequent forking and exiting of kse threads. We simply disable the penalty for thread creation and exit for kse threads.
• Cleanup the cpu estimator by using tickincr here as well. Keep ticks and ltick/ftick in the same frequency. Previously ticks were stathz and others were hz.
• Lots of new and updated comments.
• Many many others.

Tested on: up x86/amd64, 8way amd64.

An internal discussion happened in -developers@ has finally decided that we put ULE scheduler back to the “experimental” state, in order to reduce the number of reports about crashes and other bad things for it.

Currently the fact that ULE is not being actively maintained is the cause of this change. We hope that we can find someone who has interest on it. Additionally, it is worthy to have a look at David Xu’s new “CORE” scheduler (a ULE fork, using Linux’s algorithm).

今天 David Xu commit了他的新调度器(SCHED_CORE)，这个调度器基于ULE调度器，但改变了一系列算法。打算抽时间好好看一看代码。

David Xu’s two recent commits against -HEAD has finally fixed ULE on SMP, PREEMPTION and FULL_PREEMPTION case. Both his and my stress tests has proven that ULE is now rock solid again.

Now we will focus on solving other issues. Please be sure to test our next 6-STABLE snapshot and provide feedback, so we can make a great 6.0-RELEASE!

Finally we got ULE fixed! After some observation about stability this would definately be MFC’ed to RELENG_[56].

UPDATE: This is proven incomplete. We are still under investigation.

UPDATE: A subsequent commit of David Xu has finally got it fixed.

So now we have 6.0-BETA, 7.0-CURRENT, and both are frozen for bugfixes. I would struggle with the SMP ULE issue.

Scott Long has warned that RELENG_6 would be branched soon. At this point, most part of FreeBSD-CURRENT is considered to be “stable” while some of the drivers and ULE scheduler is still under revamp. Once all bugs found gets fixed we will release 6.0.

HEAD will be soft frozen until 6.0-RELEASE. I will begin to integrate some of minor changes there.

Now I got (partially) the point:

• A newly fork’ed thread grabs Giant
• Subsequently, the mutex procedure will call turnstile_wait
• However, the td->td_sched->ke_runq is NULL (means that some part of the new thread is not initialized completely)
• So boom! panic.

Interestingly this does not happen in !PREEMPTION case.

Now my question are:

1. who forks?
2. is it the child? the parent?
3. why the initialization is not completed with a wrong result?