SMP

前几天有人问 FreeBSD 的 top(1) 如何能够显示不同 CPU 的占用情况。这是 patch 过之后的输出：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

last pid:  1773;  load averages:  0.43,  0.19,  0.16                                                                                        up 0+00:34:35  16:17:04
97 processes:  1 running, 96 sleeping
CPU 0: 20.8% user,  0.0% nice,  0.0% system,  0.0% interrupt, 79.2% idle
CPU 1:  3.1% user,  0.0% nice,  0.0% system,  0.0% interrupt, 96.9% idle
Mem: 517M Active, 21M Inact, 577M Wired, 11M Cache, 48K Buf, 810M Free
Swap: 2048M Total, 2048M Free

 PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND  
 1236 delphij 1 104 0 232M 143M select 1 4:37 37.06% Xorg  
 1768 delphij 1 8 0 164M 52704K nanslp 1 0:07 36.28% kicker  
 1771 delphij 1 8 0 6212K 1664K wait 0 0:00 3.17% sh  
 1772 delphij 1 45 0 8020K 1656K select 1 0:00 2.39% ksysguardd  
 1756 delphij 1 20 0 10648K 3596K pause 1 0:00 1.17% csh  
 1754 delphij 1 4 0 11764K 1588K sbwait 1 0:00 0.78% gnome-pty-helper  
 1751 delphij 2 -8 0 136M 25756K piperd 0 0:00 0.68% gnome-terminal  

FreeBSD 8 中已经包含了这个功能。

Today, Jeff Roberson has committed his version 2.0 ULE scheduler. This new version has addressed several design issues as well as several bugs.

The new scheduler has adopted a circular queue, instead of the double-queue structure which is also found in the Linux O(1) scheduler. The latter has lead to difficulty implementing nice correctly.

For uniprocessor case, ULE is now faster.

MP algorithm has been simplified a bit.

A lot of bugfixes, etc.

To quote the original commit message:

ULE 2.0:

• Remove the double queue mechanism for timeshare threads. It was slow due to excess cache lines in play, caused suboptimal scheduling behavior with niced and other non-interactive processes, complicated priority lending, etc.
• Use a circular queue with a floating starting index for timeshare threads. Enforces fairness by moving the insertion point closer to threads with worse priorities over time.
• Give interactive timeshare threads real-time user-space priorities and place them on the realtime/ithd queue.
• Select non-interactive timeshare thread priorities based on their cpu utilization over the last 10 seconds combined with the nice value. This gives us more sane priorities and behavior in a loaded system as compared to the old method of using the interactivity score. The interactive score quickly hit a ceiling if threads were non-interactive and penalized new hog threads.
• Use one slice size for all threads. The slice is not currently dynamically set to adjust scheduling behavior of different threads.
• Add some new sysctls for scheduling parameters.

Bug fixes/Clean up:

• Fix zeroing of td_sched after initialization in sched_fork_thread() caused by recent ksegrp removal.
• Fix KSE interactivity issues related to frequent forking and exiting of kse threads. We simply disable the penalty for thread creation and exit for kse threads.
• Cleanup the cpu estimator by using tickincr here as well. Keep ticks and ltick/ftick in the same frequency. Previously ticks were stathz and others were hz.
• Lots of new and updated comments.
• Many many others.

Tested on: up x86/amd64, 8way amd64.

对于MP内核来说，这是一项用于寻找内核中存在的性能弱点的重要工具。早先kris做的性能评估看来可以继续、常规化地做下去了。

离这个Q结束还有1个月，来总结一下：

工作：

• 基本完成了内部版本的FreeBSD 6.0-RELEASE，并已开始用于生产服务器
• 替换了公司的一组基于qmail的转发服务器，采用FreeBSD 6.0和postfix
• 实现了一个验证服务
• 设计并实现了一组全新的反垃圾策略，有效地将垃圾邮件的数量降低了若干量级，并且用户无不良反映(最开始出现过一次问题，但并不是策略本身的问题)
• 设计了一组新的用户行为分析系统，并实现了其初级部分。该系统将进一步提高邮件系统的可用性
• 追和扑灭了邮件系统中的一批中等和严重程度的隐患
• 1314同学撤退后客串跑了若干次机房扑火
• 赶猪，逮老鼠，上大包……(暗语，内部人员偷着乐就行了，不要泄露国家机密;)
  (不过李逵却说……算了还是不说了)

开源软件方面的掺和：

• 为FreeBSD 6.0-RELEASE的发布进行了一些工作，总体而言得到了认可
• 翻译完了FreeBSD Architecture Handbook的SMP章节

David Xu’s two recent commits against -HEAD has finally fixed ULE on SMP, PREEMPTION and FULL_PREEMPTION case. Both his and my stress tests has proven that ULE is now rock solid again.

Now we will focus on solving other issues. Please be sure to test our next 6-STABLE snapshot and provide feedback, so we can make a great 6.0-RELEASE!

Finally we got ULE fixed! After some observation about stability this would definately be MFC’ed to RELENG_[56].

UPDATE: This is proven incomplete. We are still under investigation.

UPDATE: A subsequent commit of David Xu has finally got it fixed.

So now we have 6.0-BETA, 7.0-CURRENT, and both are frozen for bugfixes. I would struggle with the SMP ULE issue.

niklas has committed the final SMP branch code, no more SMP branchs, we have SMP in OpenBSD-CURRENT now!!

I’m going on to track down the Pentium 4 HTT issue of FreeBSD. Currently I have setup a tinderbox (hengdm’s workstation because he is not here :-P and it’s the only box which supports HTT) and compiled a SMP kernel on it.

Today I met a famous FreeBSD developer, David O’Brien (a.k.a. obrien@) and he gave us a presentation about the AMD64 architecture, which cleared my misunderstanding about the AMD64 architecture set by some marketing guys presenting in Tsinghua University.