2011年8月7日 星期日

Process Scheduling - 4

一般行程的優先順序的值是 100 (highest priority) 至 139 (lowest priority),行程一開始時會被賦予初始的優先順序,
稱為 static priority;之後在執行的期間,排程器會藉由動態增減的方式來調整優先順序,此為 dynamic priority。

real-time 行程的優先順序則是 1 (highest priority) 至 99 (lowest priority),又稱為 real-time priority。

Static priority
用來決定行程的 CPU 使用時間(base time quantum),優先順序愈高(數字愈小),執行時間也就愈長。
行程的 static priority 是繼承其父行程而來,也可以藉由設定 nice 值來做修改。


Dynamic priority
作為排程器挑選行程來執行的優先順序,dynamic priority 會在行程執行期間隨 bonus 值調整。
bonus 是一個 0 至 10 的數值;小於 5 時,表示對行程的優先權做調降;大於 5 時,則是做調升。

下表列出與行程的優先權相關的參數:


定義於 include/linux/sched.h

/*
* Priority of a process goes from 0..MAX_PRIO-1, valid RT
* priority is 0..MAX_RT_PRIO-1, and SCHED_NORMAL tasks are
* in the range MAX_RT_PRIO..MAX_PRIO-1. Priority values
* are inverted: lower p->prio value means higher priority.
*
* The MAX_USER_RT_PRIO value allows the actual maximum
* RT priority to be separate from the value exported to
* user-space. This allows kernel threads to set their
* priority to a value higher than any user task. Note:
* MAX_RT_PRIO must not be smaller than MAX_USER_RT_PRIO.
*/

#define MAX_USER_RT_PRIO  100
#define MAX_RT_PRIO      MAX_USER_RT_PRIO

#define MAX_PRIO       (MAX_RT_PRIO + 40)

#define rt_task(p)        (unlikely((p)->prio < MAX_RT_PRIO))



定義於 kernel/sched.c

/*
* Convert user-nice values [ -20 ... 0 ... 19 ]
* to static priority [ MAX_RT_PRIO..MAX_PRIO-1 ],
* and back.
*/
#define NICE_TO_PRIO(nice) (MAX_RT_PRIO + (nice) + 20)
#define PRIO_TO_NICE(prio) ((prio) - MAX_RT_PRIO - 20)
#define TASK_NICE(p)    PRIO_TO_NICE((p)->static_prio)

/*
* 'User priority' is the nice value converted to something we
* can work with better when scaling various scheduler parameters,
* it's a [ 0 ... 39 ] range.
*/
#define USER_PRIO(p)    ((p)-MAX_RT_PRIO)
#define TASK_USER_PRIO(p) USER_PRIO((p)->static_prio)
#define MAX_USER_PRIO  (USER_PRIO(MAX_PRIO))

/*
* task_timeslice() scales user-nice values [ -20 ... 0 ... 19 ]
* to time slice values: [800ms ... 100ms ... 5ms]
*
* The higher a thread's priority, the bigger timeslices
* it gets during one round of execution. But even the lowest
* priority thread gets MIN_TIMESLICE worth of execution time.
*/

#define SCALE_PRIO(x, prio) \
 max(x * (MAX_PRIO - prio) / (MAX_USER_PRIO/2), MIN_TIMESLICE)

static unsigned int task_timeslice(task_t *p)
{
 if (p->static_prio < NICE_TO_PRIO(0))
  return SCALE_PRIO(DEF_TIMESLICE*4, p->static_prio);
 else
  return SCALE_PRIO(DEF_TIMESLICE, p->static_prio);
}

/*
* effective_prio - return the priority that is based on the static
* priority but is modified by bonuses/penalties.
*
* We scale the actual sleep average [0 .... MAX_SLEEP_AVG]
* into the -5 ... 0 ... +5 bonus/penalty range.
*
* We use 25% of the full 0...39 priority range so that:
*
* 1) nice +19 interactive tasks do not preempt nice 0 CPU hogs.
* 2) nice -20 CPU hogs do not get preempted by nice 0 tasks.
*
* Both properties are important to certain workloads.
*/
static int effective_prio(task_t *p)
{
 int bonus, prio;

 if (rt_task(p))
  return p->prio;

 bonus = CURRENT_BONUS(p) - MAX_BONUS / 2;

 prio = p->static_prio - bonus;
 if (prio < MAX_RT_PRIO)
  prio = MAX_RT_PRIO;
 if (prio > MAX_PRIO-1)
  prio = MAX_PRIO-1;
 return prio;
}

/**
* task_prio - return the priority value of a given task.
* @p: the task in question.
*
* This is the priority value as seen by users in /proc.
* RT tasks are offset by -200. Normal tasks are centered
* around 0, value goes from -16 to +15.
*/
int task_prio(const task_t *p)
{
 return p->prio - MAX_RT_PRIO;
}

/**
* task_nice - return the nice value of a given task.
* @p: the task in question.
*/
int task_nice(const task_t *p)
{
 return TASK_NICE(p);
}


2011年6月11日 星期六

Process Scheduling - 3

• wait queue

相對於 run queue 的是 wait queue,這是一個存放休眠中的行程的佇列。
由型態為 wait_queue_t 的雙向鏈結串列所組成,並具有一個型態為 wait_queue_head_t 的串列首。


wait_queue_t 結構中的 flags 值為 1 時,進入休眠的行程是 exclusive process,表示只有該行程在等待特定的系統資源。
若 flags 值為 0 時,則是 nonexclusive process,表示有多個行程在等待相同的系統資源。

而 func 欄位會在初始化時設定成 default_wake_function(),這是 exclusive process 的預設喚醒函式。


wait queue head 可以靜態的宣告:

DECLARE_WAIT_QUEUE_HEAD( wait_head );

或是動態的宣告:

wait_queue_head_t wait_head;

init_waitqueue_head( &wait_head );

要讓行程休眠可以呼叫下列巨集函式,將該行程加入一個等待佇列中,並交出 CPU 使用權。

wait_event(wait_head, condition)

wait_event_interruptible(wait_head, condition)

wait_event_timeout(wait_head, condition, timeout)

wait_event_interruptible_timeout(wait_head, condition, timeout)

要喚醒休眠中的行程可以呼叫下列巨集函式,被喚醒的行程會回到執行佇列之中。

wake_up( wait_head_pt )

wake_up_interruptible( wait_head_pt )


定義於 include/linux/wait.h

typedef struct __wait_queue wait_queue_t;
typedef int (*wait_queue_func_t)(wait_queue_t *wait, unsigned mode, int sync, void *key);
int default_wake_function(wait_queue_t *wait, unsigned mode, int sync, void *key);

struct __wait_queue {
  unsigned int flags;
#define WQ_FLAG_EXCLUSIVE 0x01
  struct task_struct * task;
  wait_queue_func_t func;
  struct list_head task_list;
};

struct wait_bit_key {
  void *flags;
  int bit_nr;
};

struct wait_bit_queue {
  struct wait_bit_key key;
  wait_queue_t wait;
};

struct __wait_queue_head {
  spinlock_t lock;
  struct list_head task_list;
};
typedef struct __wait_queue_head wait_queue_head_t;

#define __wait_event(wq, condition)               \
do {                              \
  DEFINE_WAIT(__wait);                   \
                                \
  for (;;) {                          \
    prepare_to_wait(&wq, &__wait, TASK_UNINTERRUPTIBLE); \
    if (condition)                       \
      break;                       \
    schedule();                       \
  }                             \
  finish_wait(&wq, &__wait);                  \
} while (0)

#define wait_event(wq, condition)                 \
do {                              \
  if (condition)                        \
    break;                         \
  __wait_event(wq, condition);                 \
} while (0)


void FASTCALL(__wake_up(wait_queue_head_t *q, unsigned int mode, int nr, void *key));

#define wake_up(x)          __wake_up(x, TASK_UNINTERRUPTIBLE TASK_INTERRUPTIBLE, 1, NULL)
#define wake_up_interruptible(x)  __wake_up(x, TASK_INTERRUPTIBLE, 1, NULL)


定義於 kernel/wait.c

/*
* Note: we use "set_current_state()" _after_ the wait-queue add,
* because we need a memory barrier there on SMP, so that any
* wake-function that tests for the wait-queue being active
* will be guaranteed to see waitqueue addition _or_ subsequent
* tests in this thread will see the wakeup having taken place.
*
* The spin_unlock() itself is semi-permeable and only protects
* one way (it only protects stuff inside the critical region and
* stops them from bleeding out - it would still allow subsequent
* loads to move into the the critical region).
*/
void fastcall
prepare_to_wait(wait_queue_head_t *q, wait_queue_t *wait, int state)
{
  unsigned long flags;

  wait->flags &= ~WQ_FLAG_EXCLUSIVE;
  spin_lock_irqsave(&q->lock, flags);
  if (list_empty(&wait->task_list))
    __add_wait_queue(q, wait);
  /*
   * don't alter the task state if this is just going to
   * queue an async wait queue callback
   */
  if (is_sync_wait(wait))
    set_current_state(state);
  spin_unlock_irqrestore(&q->lock, flags);
}
EXPORT_SYMBOL(prepare_to_wait);


定義於 kernel/sched.c

/*
* The core wakeup function. Non-exclusive wakeups (nr_exclusive == 0) just
* wake everything up. If it's an exclusive wakeup (nr_exclusive == small +ve
* number) then we wake all the non-exclusive tasks and one exclusive task.
*
* There are circumstances in which we can try to wake a task which has already
* started to run but is not in state TASK_RUNNING. try_to_wake_up() returns
* zero in this (rare) case, and we handle it by continuing to scan the queue.
*/
static void __wake_up_common(wait_queue_head_t *q, unsigned int mode,
               int nr_exclusive, int sync, void *key)
{
  struct list_head *tmp, *next;

  list_for_each_safe(tmp, next, &q->task_list) {
    wait_queue_t *curr;
    unsigned flags;
    curr = list_entry(tmp, wait_queue_t, task_list);
    flags = curr->flags;
    if (curr->func(curr, mode, sync, key) &&
     (flags & WQ_FLAG_EXCLUSIVE) &&
     !--nr_exclusive)
      break;
  }
}

/**
* __wake_up - wake up threads blocked on a waitqueue.
* @q: the waitqueue
* @mode: which threads
* @nr_exclusive: how many wake-one or wake-many threads to wake up
*/
void fastcall __wake_up(wait_queue_head_t *q, unsigned int mode,
            int nr_exclusive, void *key)
{
  unsigned long flags;

  spin_lock_irqsave(&q->lock, flags);
  __wake_up_common(q, mode, nr_exclusive, 0, key);
  spin_unlock_irqrestore(&q->lock, flags);
}

EXPORT_SYMBOL(__wake_up);

/***
* try_to_wake_up - wake up a thread
* @p: the to-be-woken-up thread
* @state: the mask of task states that can be woken
* @sync: do a synchronous wakeup?
*
* Put it on the run-queue if it's not already there. The "current"
* thread is always on the run-queue (except when the actual
* re-schedule is in progress), and as such you're allowed to do
* the simpler "current->state = TASK_RUNNING" to mark yourself
* runnable without the overhead of this.
*
* returns failure only if the task is already active.
*/
static int try_to_wake_up(task_t *p, unsigned int state, int sync)
{
  int cpu, this_cpu, success = 0;
  unsigned long flags;
  long old_state;
  runqueue_t *rq;
#ifdef CONFIG_SMP
  unsigned long load, this_load;
  struct sched_domain *sd;
  int new_cpu;
#endif

  rq = task_rq_lock(p, &flags);
  schedstat_inc(rq, ttwu_cnt);
  old_state = p->state;
  if (!(old_state & state))
    goto out;

  if (p->array)
    goto out_running;

  cpu = task_cpu(p);
  this_cpu = smp_processor_id();

#ifdef CONFIG_SMP
  if (unlikely(task_running(rq, p)))
    goto out_activate;

  new_cpu = cpu;

  if (cpu == this_cpu unlikely(!cpu_isset(this_cpu, p->cpus_allowed)))
    goto out_set_cpu;

  load = source_load(cpu);
  this_load = target_load(this_cpu);

  /*
   * If sync wakeup then subtract the (maximum possible) effect of
   * the currently running task from the load of the current CPU:
   */
  if (sync)
    this_load -= SCHED_LOAD_SCALE;

  /* Don't pull the task off an idle CPU to a busy one */
  if (load < SCHED_LOAD_SCALE/2 && this_load > SCHED_LOAD_SCALE/2)
    goto out_set_cpu;

  new_cpu = this_cpu; /* Wake to this CPU if we can */

  /*
   * Scan domains for affine wakeup and passive balancing
   * possibilities.
   */
  for_each_domain(this_cpu, sd) {
    unsigned int imbalance;
    /*
     * Start passive balancing when half the imbalance_pct
     * limit is reached.
     */
    imbalance = sd->imbalance_pct + (sd->imbalance_pct - 100) / 2;

    if ((sd->flags & SD_WAKE_AFFINE) &&
        !task_hot(p, rq->timestamp_last_tick, sd)) {
      /*
       * This domain has SD_WAKE_AFFINE and p is cache cold
       * in this domain.
       */
      if (cpu_isset(cpu, sd->span)) {
        schedstat_inc(sd, ttwu_wake_affine);
        goto out_set_cpu;
      }
    } else if ((sd->flags & SD_WAKE_BALANCE) &&
        imbalance*this_load <= 100*load) {
      /*
       * This domain has SD_WAKE_BALANCE and there is
       * an imbalance.
       */
      if (cpu_isset(cpu, sd->span)) {
        schedstat_inc(sd, ttwu_wake_balance);
        goto out_set_cpu;
      }
    }
  }

  new_cpu = cpu; /* Could not wake to this_cpu. Wake to cpu instead */
out_set_cpu:
  schedstat_inc(rq, ttwu_attempts);
  new_cpu = wake_idle(new_cpu, p);
  if (new_cpu != cpu) {
    schedstat_inc(rq, ttwu_moved);
    set_task_cpu(p, new_cpu);
    task_rq_unlock(rq, &flags);
    /* might preempt at this point */
    rq = task_rq_lock(p, &flags);
    old_state = p->state;
    if (!(old_state & state))
      goto out;
    if (p->array)
      goto out_running;

    this_cpu = smp_processor_id();
    cpu = task_cpu(p);
  }

out_activate:
#endif /* CONFIG_SMP */
  if (old_state == TASK_UNINTERRUPTIBLE) {
    rq->nr_uninterruptible--;
    /*
     * Tasks on involuntary sleep don't earn
     * sleep_avg beyond just interactive state.
     */
    p->activated = -1;
  }

  /*
   * Sync wakeups (i.e. those types of wakeups where the waker
   * has indicated that it will leave the CPU in short order)
   * don't trigger a preemption, if the woken up task will run on
   * this cpu. (in this case the 'I will reschedule' promise of
   * the waker guarantees that the freshly woken up task is going
   * to be considered on this CPU.)
   */
  activate_task(p, rq, cpu == this_cpu);
  if (!sync cpu != this_cpu) {
    if (TASK_PREEMPTS_CURR(p, rq))
      resched_task(rq->curr);
  }
  success = 1;

out_running:
  p->state = TASK_RUNNING;
out:
  task_rq_unlock(rq, &flags);

  return success;
}

int default_wake_function(wait_queue_t *curr, unsigned mode, int sync, void *key)
{
  task_t *p = curr->task;
  return try_to_wake_up(p, mode, sync);
}

EXPORT_SYMBOL(default_wake_function);

2011年5月15日 星期日

Process Scheduling - 2

• run queue

Linux 核心將所有可執行的行程(其狀態為 TASK_RUNNING)記錄於 run queue 之中,除了 swapper 行程。
若是多處理器的系統,則每一個 CPU 會有自己的 run queue 變數。

定義於 kernel/sched.c

struct prio_array {
  unsigned int nr_active;
  unsigned long bitmap[BITMAP_SIZE];
  struct list_head queue[MAX_PRIO];
};


struct runqueue {
  spinlock_t lock;
  
  /*
   * nr_running and cpu_load should be in the same cacheline because
   * remote CPUs use both these fields when doing load calculation.
   */
  unsigned long nr_running;
#ifdef CONFIG_SMP
  unsigned long cpu_load;
#endif
  unsigned long long nr_switches;


  /*
   * This is part of a global counter where only the total sum
   * over all CPUs matters. A task can increase this counter on
   * one CPU and if it got migrated afterwards it may decrease
   * it on another CPU. Always updated under the runqueue lock:
   */
  unsigned long nr_uninterruptible;
  
  unsigned long expired_timestamp;
  unsigned long long timestamp_last_tick;
  task_t *curr, *idle;
  struct mm_struct *prev_mm;
  prio_array_t *active, *expired, arrays[2];
  int best_expired_prio;
  atomic_t nr_iowait;


#ifdef CONFIG_SMP
  struct sched_domain *sd;
  
  /* For active balancing */
  int active_balance;
  int push_cpu;
  
  task_t *migration_thread;
  struct list_head migration_queue;
#endif



#ifdef CONFIG_SCHEDSTATS
  /* latency stats */
  struct sched_info rq_sched_info;
  
  /* sys_sched_yield() stats */
  unsigned long yld_exp_empty;
  unsigned long yld_act_empty;
  unsigned long yld_both_empty;
  unsigned long yld_cnt;
  
  /* schedule() stats */
  unsigned long sched_noswitch;
  unsigned long sched_switch;
  unsigned long sched_cnt;
  unsigned long sched_goidle;
  
  /* pull_task() stats */
  unsigned long pt_gained[MAX_IDLE_TYPES];
  unsigned long pt_lost[MAX_IDLE_TYPES];
  
  /* active_load_balance() stats */
  unsigned long alb_cnt;
  unsigned long alb_lost;
  unsigned long alb_gained;
  unsigned long alb_failed;
  
  /* try_to_wake_up() stats */
  unsigned long ttwu_cnt;
  unsigned long ttwu_attempts;
  unsigned long ttwu_moved;
  
  /* wake_up_new_task() stats */
  unsigned long wunt_cnt;
  unsigned long wunt_moved;
  
  /* sched_migrate_task() stats */
  unsigned long smt_cnt;
  
  /* sched_balance_exec() stats */
  unsigned long sbe_cnt;
#endif
};


runqueue 結構中名稱為 arrays 的欄位,是具有 2 個 prio_array_t 型別(即 struct prio_array)的陣列,
arrays[0] 與 arrays[1] 都包含 140 個雙向鏈結串列的串列首,用來記錄優先等級 0 到 139 的可執行的行程。
另外有 2 個指標會分別指向 arrays[0] 與 arrays[1],代表 active 與 expired 的行程集合,如下圖所示。

在系統的運作過程中,active 行程用完其執行時間就變成 expired 行程。而當 active 的佇列為空時,
expired 的佇列就會與 active 的佇列互換,便是將這 2 個指標指向不同的 arrays 的位置來達成。

2011年5月5日 星期四

Process Scheduling - 1

Linux 核心使用 time-shared 的排程方法,並且由 Timer Interrupt 觸發。
每當計時器中斷發生時,核心會呼叫 scheduler_tick() 來重新分配 CPU 的使用權,
因此所有的行程是以分時多工的方式被執行,計時器的中斷時間間隔稱為時間片段(time-slice)。

執行程序在 Linux 中被分成兩類:

Active - 尚未用完時間片段的行程
Expired - 已經用完時間片段的行程

可執行的行程會被存放在這兩種佇列之中,而核心的排程器是從 Active 佇列的最高優先等級的程序
選出一個行程來執行,此動作為呼叫 schedule()。當行程的時間片段用完時,便會被移至 Expired 佇列。

執行佇列的優先權有 140 個級別,數字愈小表示其優先次序愈高,
0 到 99 為 real-time 行程的等級,100 到 139 為一般行程的等級(即 User mode 程式)。


上圖是一個核心排程的例子,每次 scheduler_tick() 被呼叫時,核心會取得 CPU 的使用權,
並將目前正在執行的行程的 time-slice 減一,當行程的 time-slice 用完時,核心呼叫 schedule() 挑選另一個行程來執行。
若是執行中的行程因等待某些事件而主動交出 CPU 使用權時,也會經由呼叫 schedule() 切換執行權至另一個行程。


定義於 kernel/sched.c

/*
* This function gets called by the timer code, with HZ frequency.
* We call it with interrupts disabled.
*
* It also gets called by the fork code, when changing the parent's
* timeslices.
*/
void scheduler_tick(void)
{
  int cpu = smp_processor_id();
  runqueue_t *rq = this_rq();
  task_t *p = current;

  rq->timestamp_last_tick = sched_clock();

  if (p == rq->idle) {
    if (wake_priority_sleeper(rq))
      goto out;
    rebalance_tick(cpu, rq, SCHED_IDLE);
    return;
  }

  /* Task might have expired already, but not scheduled off yet */
  if (p->array != rq->active) {
    set_tsk_need_resched(p);
    goto out;
  }
  spin_lock(&rq->lock);
  /*
   * The task was running during this tick - update the
   * time slice counter. Note: we do not update a thread's
   * priority until it either goes to sleep or uses up its
   * timeslice. This makes it possible for interactive tasks
   * to use up their timeslices at their highest priority levels.
   */
  if (rt_task(p)) {
    /*
     * RR tasks need a special form of timeslice management.
     * FIFO tasks have no timeslices.
     */
    if ((p->policy == SCHED_RR) && !--p->time_slice) {
      p->time_slice = task_timeslice(p);
      p->first_time_slice = 0;
      set_tsk_need_resched(p);

      /* put it at the end of the queue: */
      requeue_task(p, rq->active);
    }
    goto out_unlock;
  }
  if (!--p->time_slice) {
    dequeue_task(p, rq->active);
    set_tsk_need_resched(p);
    p->prio = effective_prio(p);
    p->time_slice = task_timeslice(p);
    p->first_time_slice = 0;

    if (!rq->expired_timestamp)
      rq->expired_timestamp = jiffies;
    if (!TASK_INTERACTIVE(p) EXPIRED_STARVING(rq)) {
      enqueue_task(p, rq->expired);
      if (p->static_prio < rq->best_expired_prio)
        rq->best_expired_prio = p->static_prio;
    } else
      enqueue_task(p, rq->active);
  } else {
    /*
     * Prevent a too long timeslice allowing a task to monopolize
     * the CPU. We do this by splitting up the timeslice into
     * smaller pieces.
     *
     * Note: this does not mean the task's timeslices expire or
     * get lost in any way, they just might be preempted by
     * another task of equal priority. (one with higher
     * priority would have preempted this task already.) We
     * requeue this task to the end of the list on this priority
     * level, which is in essence a round-robin of tasks with
     * equal priority.
     *
     * This only applies to tasks in the interactive
     * delta range with at least TIMESLICE_GRANULARITY to requeue.
     */
    if (TASK_INTERACTIVE(p) && !((task_timeslice(p) -
      p->time_slice) % TIMESLICE_GRANULARITY(p)) &&
      (p->time_slice >= TIMESLICE_GRANULARITY(p)) &&
      (p->array == rq->active)) {

      requeue_task(p, rq->active);
      set_tsk_need_resched(p);
    }
  }
out_unlock:
  spin_unlock(&rq->lock);
out:
  rebalance_tick(cpu, rq, NOT_IDLE);
}

/*
* schedule() is the main scheduler function.
*/
asmlinkage void __sched schedule(void)
{
  long *switch_count;
  task_t *prev, *next;
  runqueue_t *rq;
  prio_array_t *array;
  struct list_head *queue;
  unsigned long long now;
  unsigned long run_time;
  int cpu, idx;

  /*
   * Test if we are atomic. Since do_exit() needs to call into
   * schedule() atomically, we ignore that path for now.
   * Otherwise, whine if we are scheduling when we should not be.
   */
  if (likely(!current->exit_state)) {
    if (unlikely(in_atomic())) {
      printk(KERN_ERR "scheduling while atomic: "
        "%s/0x%08x/%d\n",
        current->comm, preempt_count(), current->pid);
      dump_stack();
    }
  }
  profile_hit(SCHED_PROFILING, __builtin_return_address(0));

need_resched:
  preempt_disable();
  prev = current;
  release_kernel_lock(prev);
need_resched_nonpreemptible:
  rq = this_rq();

  /*
   * The idle thread is not allowed to schedule!
   * Remove this check after it has been exercised a bit.
   */
  if (unlikely(prev == rq->idle) && prev->state != TASK_RUNNING) {
    printk(KERN_ERR "bad: scheduling from the idle thread!\n");
    dump_stack();
  }

  schedstat_inc(rq, sched_cnt);
  now = sched_clock();
  if (likely(now - prev->timestamp < NS_MAX_SLEEP_AVG))     run_time = now - prev->timestamp;
  else
    run_time = NS_MAX_SLEEP_AVG;

  /*
   * Tasks charged proportionately less run_time at high sleep_avg to
   * delay them losing their interactive status
   */
  run_time /= (CURRENT_BONUS(prev) ? : 1);

  spin_lock_irq(&rq->lock);

  if (unlikely(prev->flags & PF_DEAD))
    prev->state = EXIT_DEAD;

  switch_count = &prev->nivcsw;
  if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {
    switch_count = &prev->nvcsw;
    if (unlikely((prev->state & TASK_INTERRUPTIBLE) &&
        unlikely(signal_pending(prev))))
      prev->state = TASK_RUNNING;
    else {
      if (prev->state == TASK_UNINTERRUPTIBLE)
        rq->nr_uninterruptible++;
      deactivate_task(prev, rq);
    }
  }

  cpu = smp_processor_id();
  if (unlikely(!rq->nr_running)) {
go_idle:
    idle_balance(cpu, rq);
    if (!rq->nr_running) {
      next = rq->idle;
      rq->expired_timestamp = 0;
      wake_sleeping_dependent(cpu, rq);
      /*
       * wake_sleeping_dependent() might have released
       * the runqueue, so break out if we got new
       * tasks meanwhile:
       */
      if (!rq->nr_running)
        goto switch_tasks;
    }
  } else {
    if (dependent_sleeper(cpu, rq)) {
      next = rq->idle;
      goto switch_tasks;
    }
    /*
     * dependent_sleeper() releases and reacquires the runqueue
     * lock, hence go into the idle loop if the rq went
     * empty meanwhile:
     */
    if (unlikely(!rq->nr_running))
      goto go_idle;
  }

  array = rq->active;
  if (unlikely(!array->nr_active)) {
    /*
     * Switch the active and expired arrays.
     */
    schedstat_inc(rq, sched_switch);
    rq->active = rq->expired;
    rq->expired = array;
    array = rq->active;
    rq->expired_timestamp = 0;
    rq->best_expired_prio = MAX_PRIO;
  } else
    schedstat_inc(rq, sched_noswitch);

  idx = sched_find_first_bit(array->bitmap);
  queue = array->queue + idx;
  next = list_entry(queue->next, task_t, run_list);

  if (!rt_task(next) && next->activated > 0) {
    unsigned long long delta = now - next->timestamp;

    if (next->activated == 1)
      delta = delta * (ON_RUNQUEUE_WEIGHT * 128 / 100) / 128;

    array = next->array;
    dequeue_task(next, array);
    recalc_task_prio(next, next->timestamp + delta);
    enqueue_task(next, array);
  }
  next->activated = 0;
switch_tasks:
  if (next == rq->idle)
    schedstat_inc(rq, sched_goidle);
  prefetch(next);
  clear_tsk_need_resched(prev);
  rcu_qsctr_inc(task_cpu(prev));

  prev->sleep_avg -= run_time;
  if ((long)prev->sleep_avg <= 0)     prev->sleep_avg = 0;
  prev->timestamp = prev->last_ran = now;

  sched_info_switch(prev, next);
  if (likely(prev != next)) {
    next->timestamp = now;
    rq->nr_switches++;
    rq->curr = next;
    ++*switch_count;

    prepare_arch_switch(rq, next);
    prev = context_switch(rq, prev, next);
    barrier();

    finish_task_switch(prev);
  } else
    spin_unlock_irq(&rq->lock);

  prev = current;
  if (unlikely(reacquire_kernel_lock(prev) < 0))
    goto need_resched_nonpreemptible;
  preempt_enable_no_resched();
  if (unlikely(test_thread_flag(TIF_NEED_RESCHED)))
    goto need_resched;
}

2010年8月15日 星期日

Timing Measurements - 2

Linux 核心依據 HZ 的值設定 PIT 裝置(Intel 8254)發出中斷的頻率,並且初始化 IRQ 0 為 Timer Interrupt,其中 1193182 是 8254 晶片內部的頻率。
每當時間中斷產生時,就更新變數 jiffies_64xtime 之值。以下列出核心的相關程序的原始碼:

setup_pit_timer()     設定 PIT
time_init_hook()    初始化 IRQ 0
do_timer_interrupt()  時間中斷的處理函式

定義於 include/asm-i386/timex.h

#ifdef CONFIG_X86_ELAN
# define CLOCK_TICK_RATE 1189200 /* AMD Elan has different frequency! */
#else
# define CLOCK_TICK_RATE 1193182 /* Underlying HZ */
#endif


定義於 include/linux/jiffies.h

/* LATCH is used in the interval timer and ftape setup. */
#define LATCH ((CLOCK_TICK_RATE + HZ/2) / HZ) /* For divider */


定義於 arch/i386/kernel/timers/timer_pit.c

void setup_pit_timer(void)
{
  extern spinlock_t i8253_lock;
  unsigned long flags;

  spin_lock_irqsave(&i8253_lock, flags);
  outb_p(0x34,PIT_MODE);  /* binary, mode 2, LSB/MSB, ch 0 */
  udelay(10);
  outb_p(LATCH & 0xff , PIT_CH0); /* LSB */
  udelay(10);
  outb(LATCH >> 8 , PIT_CH0); /* MSB */
  spin_unlock_irqrestore(&i8253_lock, flags);
}


定義於 arch/i386/mach-default/setup.c

static struct irqaction irq0 = { timer_interrupt, SA_INTERRUPT, CPU_MASK_NONE, "timer", NULL, NULL};

/**
* time_init_hook - do any specific initialisations for the system timer.
*
* Description:
*  Must plug the system timer interrupt source at HZ into the IRQ listed
*  in irq_vectors.h:TIMER_IRQ
**/
void __init time_init_hook(void)
{
  setup_irq(0, &irq0);
}


定義於 include/asm-i386/mach-default/do_timer.h

/**
* do_timer_interrupt_hook - hook into timer tick
* @regs:  standard registers from interrupt
*
* Description:
*  This hook is called immediately after the timer interrupt is ack'd.
*  It's primary purpose is to allow architectures that don't possess
*  individual per CPU clocks (like the CPU APICs supply) to broadcast the
*  timer interrupt as a means of triggering reschedules etc.
**/

static inline void do_timer_interrupt_hook(struct pt_regs *regs)
{
  do_timer(regs);
#ifndef CONFIG_SMP
  update_process_times(user_mode(regs));
#endif
/*
* In the SMP case we use the local APIC timer interrupt to do the
* profiling, except when we simulate SMP mode on a uniprocessor
* system, in that case we have to call the local interrupt handler.
*/
#ifndef CONFIG_X86_LOCAL_APIC
  profile_tick(CPU_PROFILING, regs);
#else
  if (!using_apic_timer)
    smp_local_timer_interrupt(regs);
#endif
}


定義於 arch/i386/kernel/time.c

/*
* Called by the timer interrupt. xtime_lock must already be taken
* by the timer IRQ!
*/
static inline void update_times(void)
{
  unsigned long ticks;

  ticks = jiffies - wall_jiffies;
  if (ticks) {
    wall_jiffies += ticks;
    update_wall_time(ticks);
  }
  calc_load(ticks);
}

/*
* The 64-bit jiffies value is not atomic - you MUST NOT read it
* without sampling the sequence number in xtime_lock.
* jiffies is defined in the linker script...
*/

void do_timer(struct pt_regs *regs)
{
  jiffies_64++;
  update_times();
}

/*
* timer_interrupt() needs to keep up the real-time clock,
* as well as call the "do_timer()" routine every clocktick
*/
static inline void do_timer_interrupt(int irq, void *dev_id,
          struct pt_regs *regs)
{
#ifdef CONFIG_X86_IO_APIC
  if (timer_ack) {
    /*
     * Subtle, when I/O APICs are used we have to ack timer IRQ
     * manually to reset the IRR bit for do_slow_gettimeoffset().
     * This will also deassert NMI lines for the watchdog if run
     * on an 82489DX-based system.
     */
    spin_lock(&i8259A_lock);
    outb(0x0c, PIC_MASTER_OCW3);
    /* Ack the IRQ; AEOI will end it automatically. */
    inb(PIC_MASTER_POLL);
    spin_unlock(&i8259A_lock);
  }
#endif

  do_timer_interrupt_hook(regs);

  /*
   * If we have an externally synchronized Linux clock, then update
   * CMOS clock accordingly every ~11 minutes. Set_rtc_mmss() has to be
   * called as close as possible to 500 ms before the new second starts.
   */
  if ((time_status & STA_UNSYNC) == 0 &&
   xtime.tv_sec > last_rtc_update + 660 &&
   (xtime.tv_nsec / 1000)
      >= USEC_AFTER - ((unsigned) TICK_SIZE) / 2 &&
   (xtime.tv_nsec / 1000)
      <= USEC_BEFORE + ((unsigned) TICK_SIZE) / 2) {
    /* horrible...FIXME */
    if (efi_enabled) {
      if (efi_set_rtc_mmss(xtime.tv_sec) == 0)
        last_rtc_update = xtime.tv_sec;
      else
        last_rtc_update = xtime.tv_sec - 600;
    } else if (set_rtc_mmss(xtime.tv_sec) == 0)
      last_rtc_update = xtime.tv_sec;
    else
      last_rtc_update = xtime.tv_sec - 600; /* do it again in 60 s */
  }

  if (MCA_bus) {
    /* The PS/2 uses level-triggered interrupts. You can't
    turn them off, nor would you want to (any attempt to
    enable edge-triggered interrupts usually gets intercepted by a
    special hardware circuit). Hence we have to acknowledge
    the timer interrupt. Through some incredibly stupid
    design idea, the reset for IRQ 0 is done by setting the
    high bit of the PPI port B (0x61). Note that some PS/2s,
    notably the 55SX, work fine if this is removed. */

    irq = inb_p( 0x61 ); /* read the current state */
    outb_p( irq0x80, 0x61 ); /* reset the IRQ */
  }
}

/*
* This is the same as the above, except we _also_ save the current
* Time Stamp Counter value at the time of the timer interrupt, so that
* we later on can estimate the time of day more exactly.
*/
irqreturn_t timer_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
  /*
   * Here we are in the timer irq handler. We just have irqs locally
   * disabled but we don't know if the timer_bh is running on the other
   * CPU. We need to avoid to SMP race with it. NOTE: we don' t need
   * the irq version of write_lock because as just said we have irq
   * locally disabled. -arca
   */
  write_seqlock(&xtime_lock);

  cur_timer->mark_offset();

  do_timer_interrupt(irq, NULL, regs);

  write_sequnlock(&xtime_lock);
  return IRQ_HANDLED;
}

2010年8月14日 星期六

Timing Measurements - 1

System Time and Date

Linux 的系統時間與日期是靠一顆 Real Time Clock (RTC) 的晶片所維護的,應用程式可經由 /dev/rtc 來控制晶片,
核心則是透過 I/O port 0x700x71 來控制。


Time Measurement

x86 處理器有一名為 Time Stamp Counter (TSC) 的暫存器,每當 CPU 的 clock 腳位有訊號時,TSC 就會遞增。
若 clock 的頻率是 1 GHz,則 TSC 會每 1 nanosecond 做加 1 的動作。Linux 核心使用 TSC 的值便可讓做出精準的時間量測。


Timer Interrupt

PC 硬體上會有至少一個的 Programmable Interval Timer (PIT) 裝置,如 Intel 的 8254 晶片,提供定時的功能,
並且於 timeout 時對 CPU 發出中斷通知,也就是 Timer Interrupt - IRQ 0
PIT 可以透過 I/O port 0x400x43 的 4 個埠口來控制。


tick, jiffies and xtime

Linux 核心設定 PIT 產生頻率為 1000 Hz 的 Timer Interrupt,即每 1 millisecond 會產生一次中斷,
此 1 millisecond 的時間稱為一個 "tick",該值存於變數 tick_nsec 之中,而頻率值以巨集 HZ 表示。

愈短的 tick 值會得到快速的 I/O 多工的反應時間,但亦會拖慢應用程式的執行速度。
通常較慢的機器會設定 10 millisecond 的 tick 值,而較快的機器會設定 1 millisecond 的 tick 值。

定義於 kernel/timer.c

/*
* Timekeeping variables
*/
unsigned long tick_usec = TICK_USEC;  /* USER_HZ period (usec) */
unsigned long tick_nsec = TICK_NSEC;  /* ACTHZ period (nsec) */

/*
* The current time
* wall_to_monotonic is what we need to add to xtime (or xtime corrected
* for sub jiffie times) to get to monotonic time. Monotonic is pegged
* at zero at system boot time, so wall_to_monotonic will be negative,
* however, we will ALWAYS keep the tv_nsec part positive so we can use
* the usual normalization.
*/
struct timespec xtime __attribute__ ((aligned (16)));
struct timespec wall_to_monotonic __attribute__ ((aligned (16)));

EXPORT_SYMBOL(xtime);


定義於 include/asm-i386/param.h

#ifdef __KERNEL__
# define HZ      1000    /* Internal kernel timer frequency */
# define USER_HZ  100    /* .. some user interfaces are in "ticks" */
# define CLOCKS_PER_SEC (USER_HZ) /* like times() */
#endif

#ifndef HZ
#define HZ 100
#endif

核心變數 jiffies 用來儲存自系統啟動以來,共經過了多少次 tick 的值。故每次的 tick 就會使 jiffies 遞增。
由於 jiffies 的長度只有 32-bit,對於 tick = 1 millisecond 機器,經過約 50 天就會讓 jiffies 溢位。
因此真正用來儲存 jiffies 值的是長度為 64-bit 的變數 jiffies_64

定義於 include/linux/jiffies.h

/*
* The 64-bit value is not volatile - you MUST NOT read it
* without sampling the sequence number in xtime_lock.
* get_jiffies_64() will do this for you as appropriate.
*/
extern u64 __jiffy_data jiffies_64;
extern unsigned long volatile __jiffy_data jiffies;

核心變數 xtime 用來記錄自系目前的時間與日期,其結構包含兩個欄位:
tv_sec  儲存自 1970.01.01 以來,共經過多少秒 (second)
• tv_nsec 儲存自上一秒的時間以來,共經過多少的奈秒 (nanosecond)

定義於 include/linux/time.h

#ifndef _STRUCT_TIMESPEC
#define _STRUCT_TIMESPEC
struct timespec {
  time_t tv_sec;   /* seconds */
  long   tv_nsec;  /* nanoseconds */
};
#endif /* _STRUCT_TIMESPEC */

2010年3月1日 星期一

Linux Synchronization - 4

"scheduling while atomic: x/y/z"

其中 x 為行程的名稱 (current->comm)
   y 為16進制的 preempt_count
   z 為 PID

出現此訊息時,表示目前的 kernel task 是一個不可被中斷的行程,但卻被排程進入休眠。
以下為核心列印該錯誤訊息的程式片段:

/*
* schedule() is the main scheduler function.
*/
asmlinkage void __sched schedule(void)
{
  long *switch_count;
  task_t *prev, *next;
  runqueue_t *rq;
  prio_array_t *array;
  struct list_head *queue;
  unsigned long long now;
  unsigned long run_time;
  int cpu, idx;

  /*
  * Test if we are atomic. Since do_exit() needs to call into
  * schedule() atomically, we ignore that path for now.
  * Otherwise, whine if we are scheduling when we should not be.
  */
  if (likely(!current->exit_state)) {
    if (unlikely(in_atomic())) {
      printk(KERN_ERR "scheduling while atomic: "
        "%s/0x%08x/%d\n",
        current->comm, preempt_count(), current->pid);
      dump_stack();
    }
  }
  profile_hit(SCHED_PROFILING, __builtin_return_address(0));

  ... /* 以下略 */
}

定義於 kernel/sched.c

#if defined(CONFIG_PREEMPT) && !defined(CONFIG_PREEMPT_BKL)
# define in_atomic()  ((preempt_count() & ~PREEMPT_ACTIVE) != kernel_locked())
#else
# define in_atomic()  ((preempt_count() & ~PREEMPT_ACTIVE) != 0)
#endif

定義於 include/linux/hardirq.h

#define preempt_count()  (current_thread_info()->preempt_count)

定義於 include/linux/preempt.h

當核心要進行 schedule 時,會先檢查目前是否為不可中斷的行程(in_atomic)
若為真,則印出 "scheduling while atomic" 的訊息,表示此時不應該做排程

而 in_atomic 則是判斷 kernel preemption 是否為關閉的狀態
也就是關閉 preemption 是為了讓行程能夠不被打斷的執行,直到完成其工作
換言之,開啟 preemption 就允許其他行程取代目前行程,故可進行排程


以下的程式片段在定義 CONFIG_PREEMPT 的核心會造成 "scheduling while atomic" 錯誤:
(若未定義 CONFIG_PREEMPT 就不是 in_atomic 的程序)

 struct semaphore _sem;
 spinlock_t _lock;

 spin_lock_init( &_lock );
 sema_init(&_sem, 0);

 spin_lock( &_lock );
 down_interruptible( &_sem ); /* 在 Spin lock 中休眠 */
 spin_unlock( &_lock );



2010年2月25日 星期四

Linux Synchronization - 3

Spin lock 的變化型:

spin_lock(spinlock_t *lock) & spin_unlock(spinlock_t *lock)
最基本的 Spin lock 函式,用來取得或釋放 lock。適用於多處理器系統,且共用資源會被不同的 CPU 執行的行程同時存取的情況。


spin_lock_bh(spinlock_t *lock) & spin_unlock_bh(spinlock_t *lock)
加上 Local softirq disabling 的 Spin lock 函式,關閉 Bottom half 的程序並取得 lock,或是釋放 lock 並開啟 Bottom half 的程序。
適用於多處理器系統,且共用資源會被 Bottom half 的程序所存取的情況。

spin_lock_bh(l) = local_bh_disable() + spin_lock(l)
spin_unlock_bh(l) = spin_unlock(l) + local_bh_enable()


spin_lock_irq(spinlock_t *lock) & spin_unlock_irq(spinlock_t *lock)
加上 Local interrupt disabling 的 Spin lock 函式,關閉 CPU 的中斷並取得 lock,或是釋放 lock 並開啟 CPU 的中斷。
適用於多處理器系統,且共用資源會被中斷處理函式所存取的情況。

spin_lock_irq(l) = local_irq_disable() + spin_lock(l)
spin_unlock_irq(l) = spin_unlock(l) + local_irq_enable()


spin_lock_irqsave(spinlock_t *lock, unsigned long flags) & spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)
加上 Local interrupt disabling 的 Spin lock 函式,關閉 CPU 的中斷並取得 lock,或是釋放 lock 並開啟 CPU 的中斷。
此外,在關閉中斷前會儲存中斷的狀態,以及使用該儲存的狀態來開啟中斷。
適用於多處理器系統,且共用資源會被中斷處理函式所存取,同時中斷會在呼叫 Spin lock 之前就己經關閉的場合。
這一組 spin lock 會有當釋放 lock 時,不會再啟動中斷的效果(例如巢狀的中斷)。

spin_lock_irqsave(l, f) = local_irq_save(f) + spin_lock(l)
spin_unlock_irqrestore(l, f) = spin_unlock(l) + local_irq_restore(f)


使用 Spin lock 的注意事項:

握有 lock 的行程不能進入休眠狀態或是被另一行程搶走 CPU 執行權,否則可能會讓其他想要取得同一個 lock 的行程
進入無窮的 "忙碌迴圈" 之中 ,CPU資源就會完全地被該行程給吃掉(因為握有 lock 的行程不知何時才會釋放 lock)。

2010年2月21日 星期日

Linux Synchronization - 2

以下為 Spin lock 不同的實作方式:

1. Uni-processor 且為 Non-preemptive 的系統 (CONFIG_SMP 及 CONFIG_PREEMPT 均未定義)
Spin lock 完全無作用——因為在這種系統中,共用資源在同一時間不可能會被兩個以上的行程所存取,故不需要實作出 lock 的保護功能。

typedef struct { } spinlock_t; /* 內容為空的結構 */

#define spin_lock(lock)   _spin_lock(lock)
#define spin_unlock(lock)  _spin_unlock(lock)
#define _spin_lock(lock) \

  do { \
    preempt_disable(); \
    _raw_spin_lock(lock); \
    __acquire(lock); \
  } while(0)

#define _spin_unlock(lock) \
  do { \
    _raw_spin_unlock(lock); \
    preempt_enable(); \
    __release(lock); \
  } while (0)

#define _raw_spin_lock(lock)   do { (void)(lock); } while(0)
#define _raw_spin_unlock(lock)  do { (void)(lock); } while(0)

定義於 include/linux/spinlock.h

#define preempt_disable()  do { } while (0)
#define preempt_enable()  do { } while (0)

定義於 include/linux/preempt.h

#define __acquire(x)  (void)0
#define __release(x)  (void)0

定義於 include/linux/compiler.h


2. Uni-processor 且為 Preemptive 的系統 (定義 CONFIG_PREEMPT 且 CONFIG_SMP 未定義)
Spin lock 實作成開、關 Kernel preemption 的功能,目的也是讓共用資源在同一時間不會被兩個以上的行程所存取。

typedef struct { } spinlock_t; /* 內容為空的結構 */

#define spin_lock(lock)   _spin_lock(lock)
#define spin_unlock(lock)  _spin_unlock(lock)
#define _spin_lock(lock) \

  do { \
    preempt_disable(); \
    _raw_spin_lock(lock); \
    __acquire(lock); \
  } while(0)

#define _spin_unlock(lock) \
  do { \
    _raw_spin_unlock(lock); \
    preempt_enable(); \
    __release(lock); \
  } while (0)

#define _raw_spin_lock(lock)   do { (void)(lock); } while(0)
#define _raw_spin_unlock(lock)  do { (void)(lock); } while(0)


定義於 include/linux/spinlock.h

#define preempt_disable() \
  do { \
    inc_preempt_count(); \
    barrier(); \
  } while (0)

#define preempt_enable() \
  do { \
    preempt_enable_no_resched(); \
    preempt_check_resched(); \
  } while (0)

定義於 include/linux/preempt.h

#define __acquire(x)  (void)0
#define __release(x)  (void)0

定義於 include/linux/compiler.h


3. Multi-processor 且為 Non-preemptive 的系統 (定義 CONFIG_SMP 且 CONFIG_PREEMPT 未定義)
Spin lock 有實作出 "忙碌迴圈" 的功能,用來鎖住想要使用共用資源卻又無法取得 lock 的行程。

typedef struct {
  volatile unsigned int slock;
} spinlock_t;

static inline void _raw_spin_lock(spinlock_t *lock)
{
  __asm__ __volatile__(
    spin_lock_string
    :"=m" (lock->slock) : : "memory"
  );
}

static inline void _raw_spin_unlock(spinlock_t *lock)
{
  char oldval = 1;
  __asm__ __volatile__(
    spin_unlock_string
  );
}

#define spin_lock_string \
  "\n1:\t" \
  "lock ; decb %0\n\t" \
  "jns 3f\n" \
  "2:\t" \
  "rep;nop\n\t" \
  "cmpb $0,%0\n\t" \
  "jle 2b\n\t" \
  "jmp 1b\n" \
  "3:\n\t"

#define spin_unlock_string \
  "xchgb %b0, %1" \
    :"=q" (oldval), "=m" (lock->slock) \
    :"0" (oldval) : "memory"


定義於 include/asm-i386/spinlock.h

#define spin_lock(lock)   _spin_lock(lock)
#define spin_unlock(lock)  _spin_unlock(lock)

定義於 include/linux/spinlock.h

void __lockfunc _spin_lock(spinlock_t *lock)
{
  preempt_disable();
  _raw_spin_lock(lock);
}

EXPORT_SYMBOL(_spin_lock);

void __lockfunc _spin_unlock(spinlock_t *lock)
{
  _raw_spin_unlock(lock);
  preempt_enable();
}

EXPORT_SYMBOL(_spin_unlock);

定義於 kernel/spinlock.c

#define preempt_disable()  do { } while (0)
#define preempt_enable()  do { } while (0)

定義於 include/linux/preempt.h

4. Multi-processor 且為 Preemptive 的系統 (定義 CONFIG_SMP 及 CONFIG_PREEMPT)
Spin lock 有實作出 "忙碌迴圈" 的功能,而且被鎖住的行程,可以讓其他行程搶走 CPU 的執行權。

typedef struct {
  volatile unsigned int slock;
  unsigned int break_lock; /* #ifdef CONFIG_PREEMPT */
} spinlock_t;

static inline int _raw_spin_trylock(spinlock_t *lock)
{
  char oldval;
  __asm__ __volatile__(
    "xchgb %b0,%1"
    :"=q" (oldval), "=m" (lock->slock)
    :"0" (0) : "memory"
  );
  return oldval > 0;
}

static inline void _raw_spin_unlock(spinlock_t *lock)
{
  char oldval = 1;
  __asm__ __volatile__(
    spin_unlock_string
  );
}

#define spin_unlock_string \
  "xchgb %b0, %1" \
    :"=q" (oldval), "=m" (lock->slock) \
    :"0" (oldval) : "memory"

#define spin_is_locked(x)  (*(volatile signed char *)(&(x)->slock) <= 0)


定義於 include/asm-i386/spinlock.h

#define spin_lock(lock)    _spin_lock(lock)
#define spin_unlock(lock)   _spin_unlock(lock)
#define spin_can_lock(lock)  (!spin_is_locked(lock))

定義於 include/linux/spinlock.h

#define BUILD_LOCK_OPS(op, locktype) \
void __lockfunc _##op##_lock(locktype##_t *lock) \
{ \
  preempt_disable(); \
  for (;;) { \
    if (likely(_raw_##op##_trylock(lock))) \
      break; \
    preempt_enable(); \
    if (!(lock)->break_lock) \
      (lock)->break_lock = 1; \
    while (!op##_can_lock(lock) && (lock)->break_lock) \
      cpu_relax(); \
    preempt_disable(); \
  } \
} \
\
EXPORT_SYMBOL(_##op##_lock);

BUILD_LOCK_OPS(spin, spinlock); /* #ifdef CONFIG_PREEMPT */

void __lockfunc _spin_unlock(spinlock_t *lock)
{
  _raw_spin_unlock(lock);
  preempt_enable();
}

EXPORT_SYMBOL(_spin_unlock);


定義於 kernel/spinlock.c

#define preempt_disable() \
  do { \
    inc_preempt_count(); \
    barrier(); \
  } while (0)

#define preempt_enable() \
  do { \
    preempt_enable_no_resched(); \
    preempt_check_resched(); \
  } while (0)


定義於 include/linux/preempt.h

2010年2月1日 星期一

Linux Synchronization - 1

以下列出 Kernel 的同步機制:

1. Per-CPU variables
將 Kernel 中的變數宣告成 per-CPU 的陣列型式,每一個 CPU 都有屬於自己的變數空間,不會和其他的CPU共用。

2. Atomic operations
讓存取記憶體的指令能夠完整的執行完成,不會被中斷或其他的 CPU 指令所打斷。

3. Memory barriers
利用一些 Macro 安插於某段程式中,確保程式能如預期的順序來執行。

4. Spin lock
以 lock 的方法來保護特定資源只能被一個程序使用,其他欲使用該資源的程序會進入 "忙碌迴圈" 等待資源被釋放為止。

5. Semaphore
類似 Spin lock 的保護方法,但待等特定資源的行程會進入休眠狀態,也就是要讓出 CPU 使用權,因此不可用於中斷處理函式。

6. Seqlock
類似 Spin lock 的原理,將 lock 分為 reader 及 writer 兩種,分別用來保護資源被謮與寫的動作。而 writer 有高於 reader 的優先權,亦即同時發生讀、寫同一資源時,寫的動作會優先執行。

7. Local interrupt disabling
利用關閉目前 CPU 的中斷處理函式來保護程序不會被其他中斷所打斷,但是在多 CPU 的機器上不能使其他 CPU 關閉中斷。

8. Local softirq disabling
利用停止延遲執行的中斷處理函式(例如 Bottom Half),來保護特定資源不會被其他的延遲執行的中斷處理函式所存取,而且此種方式不會關閉中斷。

9. Read-Copy Update (RCU)
利用指標的資料結構,當某指標發生被寫的動作時,將該指標複製一份(例如拷貝指標所指向的記憶體),然後對複本進行存取。完成寫的動作後,再更新該指標成為修改過的複本(例如將原來的指標改成複製後的記憶體位址)。

下表列出如何使用同步機制來保護 Kernel data structure:

2010年1月31日 星期日

Linux Interrupts and Exceptions - 5

Kernel 使用 CPU 資源的策略如下:

1. 若 CPU 在執行 User mode 程式時有中斷發生,則切換 CPU 執行權去處理中斷要求。

2. 若 CPU 在執行 Kernel mode 程式時有中斷發生,則切換 CPU 執行權去處理中斷要求。

3. 若 CPU 在處理中斷服務時有另一個中斷發生,則切換 CPU 執行權去處理新的中斷要求,接著再繼續原來的中斷服務。

4. 中斷請求結束後,CPU 可能會去處理其他的 Kernel mode 程式,而不是原來在中斷發生前正在處理的 Kernel mode 程式。

以上規則 1~3 就是核心對巢狀 Exception 及 Interrupt 的處理方式,而規則 4 則是 Kernel preemption 的特性—在 Kernel mode 中,
一個正在執行的 process 可以被另一個 process (有較高的執行優先權) 所取代,也就是行程可搶奪 CPU 的使用權。
具有此特性的核心適用於硬體動作頻繁的場合,可使得正在等待 I/O 動作的行程讓出 CPU。

2009年7月3日 星期五

Linux Interrupts and Exceptions - 4

當 I/O Interrupt 發生時,CPU 從 IDT 表中找出該中斷的位置,並取得記錄於 IDT 的 Segment SelectorOffset
再利用這兩個值將目前指令跳至 Interrupt Handler,如下圖所示:


Interrupt Handler 的動作即是呼叫 do_IRQ() 函式來處理該中斷,其 IRQ Number 由引數 struct pt_regs 傳入。

定義於 include/asm-i386/ptrace.h

/* this struct defines the way the registers are stored on the
stack during a system call. */
struct pt_regs {
  long ebx;
  long ecx;
  long edx;
  long esi;
  long edi;
  long ebp;
  long eax;
  int xds;
  int xes;
  long orig_eax;
  long eip;
  int xcs;
  long eflags;
  long esp;
  int xss;
};

定義於 arch/i386/kernel/irq.c

/*
* do_IRQ handles all normal device IRQ's (the special
* SMP cross-CPU interrupts have their own specific
* handlers).
*/
fastcall unsigned int do_IRQ(struct pt_regs *regs){
  /* high bits used in ret_from_ code */
  int irq = regs->orig_eax & 0xff;
#ifdef CONFIG_4KSTACKS
  union irq_ctx *curctx, *irqctx;
  u32 *isp;#endif
  irq_enter();
#ifdef CONFIG_DEBUG_STACKOVERFLOW
  /* Debugging check for stack overflow: is there less than 1KB free? */
  {
    long esp;
    __asm__ __volatile__("andl %%esp,%0" :
          "=r" (esp) : "0" (THREAD_SIZE - 1));
    if (unlikely(esp < (sizeof(struct thread_info) + STACK_WARN))) {
      printk("do_IRQ: stack overflow: %ld\n",
        esp - sizeof(struct thread_info));
      dump_stack();
    }
  }
#endif

#ifdef CONFIG_4KSTACKS
  curctx = (union irq_ctx *) current_thread_info();
  irqctx = hardirq_ctx[smp_processor_id()];

  /*
   * this is where we switch to the IRQ stack. However, if we are
   * already using the IRQ stack (because we interrupted a hardirq
   * handler) we can't do that and just have to keep using the
   * current stack (which is the irq stack already after all)
   */
  if (curctx != irqctx) {
    int arg1, arg2, ebx;
    /* build the stack frame on the IRQ stack */
    isp = (u32*) ((char*)irqctx + sizeof(*irqctx));
    irqctx->tinfo.task = curctx->tinfo.task;
    irqctx->tinfo.previous_esp = current_stack_pointer;
    asm volatile(
      "  xchgl %%ebx,%%esp  \n"
      "  call __do_IRQ   \n"
      "  movl %%ebx,%%esp  \n"
      : "=a" (arg1), "=d" (arg2), "=b" (ebx)
      : "0" (irq), "1" (regs), "2" (isp)
      : "memory", "cc", "ecx"
    );
  } else
#endif

  __do_IRQ(irq, regs);

  irq_exit();

  return 1;
}

定義於 kernel/irq/handle.c

/*
* do_IRQ handles all normal device IRQ's (the special
* SMP cross-CPU interrupts have their own specific
* handlers).
*/
fastcall unsigned int __do_IRQ(unsigned int irq, struct pt_regs *regs)
{
  irq_desc_t *desc = irq_desc + irq;
  struct irqaction * action;
  unsigned int status;

  kstat_this_cpu.irqs[irq]++;
  if (desc->status & IRQ_PER_CPU) {
    irqreturn_t action_ret;

    /*
     * No locking required for CPU-local interrupts:
     */
    desc->handler->ack(irq);
    action_ret = handle_IRQ_event(irq, regs, desc->action);
    if (!noirqdebug)
      note_interrupt(irq, desc, action_ret);
    desc->handler->end(irq);
    return 1;
  }

  spin_lock(&desc->lock);
  desc->handler->ack(irq);
  /*
   * REPLAY is when Linux resends an IRQ that was dropped earlier
   * WAITING is used by probe to mark irqs that are being tested
   */
  status = desc->status & ~(IRQ_REPLAY IRQ_WAITING);
  status = IRQ_PENDING; /* we _want_ to handle it */

  /*
   * If the IRQ is disabled for whatever reason, we cannot
   * use the action we have.
   */
  action = NULL;
  if (likely(!(status & (IRQ_DISABLED IRQ_INPROGRESS)))) {
    action = desc->action;
    status &= ~IRQ_PENDING; /* we commit to handling */
    status = IRQ_INPROGRESS; /* we are handling it */
  }
  desc->status = status;

  /*
   * If there is no IRQ handler or it was disabled, exit early.
   * Since we set PENDING, if another processor is handling
   * a different instance of this same irq, the other processor
   * will take care of it.
   */
  if (unlikely(!action))
    goto out;

  /*
   * Edge triggered interrupts need to remember
   * pending events.
   * This applies to any hw interrupts that allow a second
   * instance of the same irq to arrive while we are in do_IRQ
   * or in the handler. But the code here only handles the _second_
   * instance of the irq, not the third or fourth. So it is mostly
   * useful for irq hardware that does not mask cleanly in an
   * SMP environment.
   */
  for (;;) {
    irqreturn_t action_ret;

    spin_unlock(&desc->lock);

    action_ret = handle_IRQ_event(irq, regs, action);

    spin_lock(&desc->lock);
    if (!noirqdebug)
      note_interrupt(irq, desc, action_ret);
    if (likely(!(desc->status & IRQ_PENDING)))
      break;
    desc->status &= ~IRQ_PENDING;
  }
  desc->status &= ~IRQ_INPROGRESS;

out:
  /*
   * The ->end() handler has to deal with interrupts which got
   * disabled while the handler was running.
   */
  desc->handler->end(irq);
  spin_unlock(&desc->lock);

  return 1;
}

2009年3月17日 星期二

Linux Interrupts and Exceptions - 3

I/O Interrupt (硬體裝置的中斷) 處理函式是以 request_irq() 來向核心註冊的,並且以 free_irq() 來清除

定義於 include/linux/interrupt.h

int request_irq(
    unsigned int  irq,
    irqreturn_t   (*handler)(int, void *, struct pt_regs *),
    unsigned long  irqflags,
    const char   *devname,
    void      *dev_id
  );

void free_irq(
    unsigned int irq,
    void     *dev_id
  );

struct irqaction {
  irqreturn_t (*handler)(int, void *, struct pt_regs *);
  unsigned long flags;
  cpumask_t mask;
  const char *name;
  void *dev_id;
  struct irqaction *next;
  int irq;
  struct proc_dir_entry *dir;
};

request_irq() 函式內部則是呼叫 setup_irq() 來進行 IRQ Line 的註冊
其動作是將要求中斷處理的資訊包裝成 struct irqaction 的結構,然後記錄於 irq_desc 陣列中的 IRQ Line 的位置
若有共用 IRQ Line 的中斷,則會以鏈結串列的方式掛在同一個陣列的位置上,如下圖所示

圖中的 hw_interrupt_type 是一個描述中斷控制器的資料結構,用來抽象化不同的中斷控制硬體
例如:舊式的 8259A,或新式的 APIC,也可能是無任何中斷控制器

定義於 include/linux/irq.h

/*
* Interrupt controller descriptor. This is all we need
* to describe about the low-level hardware.
*/
struct hw_interrupt_type {
  const char * typename;
  unsigned int (*startup)(unsigned int irq);
  void (*shutdown)(unsigned int irq);
  void (*enable)(unsigned int irq);
  void (*disable)(unsigned int irq);
  void (*ack)(unsigned int irq);
  void (*end)(unsigned int irq);
  void (*set_affinity)(unsigned int irq, cpumask_t dest);
};

typedef struct hw_interrupt_type hw_irq_controller;

定義於 arch/i386/kernel/i8259.c

static struct hw_interrupt_type i8259A_irq_type = {
  "XT-PIC",
  startup_8259A_irq,
  shutdown_8259A_irq,
  enable_8259A_irq,
  disable_8259A_irq,
  mask_and_ack_8259A,
  end_8259A_irq,
  NULL
};

定義於 arch/i386/kernel/io_apic.c

/*
* Level and edge triggered IO-APIC interrupts need different handling,
* so we use two separate IRQ descriptors. Edge triggered IRQs can be
* handled with the level-triggered descriptor, but that one has slightly
* more overhead. Level-triggered interrupts cannot be handled with the
* edge-triggered handler, without risking IRQ storms and other ugly
* races.
*/
static struct hw_interrupt_type ioapic_edge_type = {
  .typename  = "IO-APIC-edge",
  .startup   = startup_edge_ioapic,
  .shutdown  = shutdown_edge_ioapic,
  .enable    = enable_edge_ioapic,
  .disable   = disable_edge_ioapic,
  .ack     = ack_edge_ioapic,
  .end    = end_edge_ioapic,
  .set_affinity = set_ioapic_affinity,
};

static struct hw_interrupt_type ioapic_level_type = {
  .typename   = "IO-APIC-level",
  .startup   = startup_level_ioapic,
  .shutdown   = shutdown_level_ioapic,
  .enable    = enable_level_ioapic,
  .disable   = disable_level_ioapic,
  .ack     = mask_and_ack_level_ioapic,
  .end     = end_level_ioapic,
  .set_affinity = set_ioapic_affinity,
};

定義於 kernel/irq/handle.c

struct hw_interrupt_type no_irq_type = {
  .typename = "none",
  .startup = startup_none,
  .shutdown = shutdown_none,
  .enable = enable_none,
  .disable = disable_none,
  .ack = ack_none,
  .end = end_none,
  .set_affinity = NULL
};

2009年3月6日 星期五

Linux Interrupts and Exceptions - 2

下圖為 x86 處理器的 256 個中斷配置,其中 vector 編號 0-19 是前面提過的 Exception
編號 32-127 是硬體裝置的 I/O Interrupt,而編號 128 則是由 Linux 核心作為 System Call 之用


在 PC 架構中,有幾個裝置的中斷訊號必須分配到中斷控制器的固定 IRQ Line
如下圖的 IRQ 0、2 及 13
IRQ 欄位表示硬體的 IRQ Line,INT 欄位表示 Linux 核心使用的 IRQ Number

上述的 I/O Interrupt 及其處理函式被儲存於 Linux 核心的 irq_desc 陣列中
變數型態為 struct irq_desc (或 irq_desc_t),陣列大小為 NR_IRQS

定義於 kernel/irq/handle.c

irq_desc_t irq_desc[NR_IRQS] __cacheline_aligned = {
  [0 ... NR_IRQS-1] = {
    .handler = &no_irq_type,
    .lock = SPIN_LOCK_UNLOCKED
  }
};


定義於 include/linux/irq.h

typedef struct irq_desc {
  hw_irq_controller *handler;
  void *handler_data;
  struct irqaction *action;  /* IRQ action list */
  unsigned int status;    /* IRQ status */
  unsigned int depth;    /* nested irq disables */
  unsigned int irq_count;  /* For detecting broken interrupts */
  unsigned int irqs_unhandled;
  spinlock_t lock;
} ____cacheline_aligned irq_desc_t;


定義於 include/asm-i386/mach-default/irq_vectors_limits.h

#ifdef CONFIG_X86_IO_APIC
#define NR_IRQS 224
# if (224 >= 32 * NR_CPUS)
# define NR_IRQ_VECTORS NR_IRQS
# else
# define NR_IRQ_VECTORS (32 * NR_CPUS)
# endif
#else
#define NR_IRQS 16
#define NR_IRQ_VECTORS NR_IRQS
#endif


以上可看出,若系統使用 APIC 來管理裝置中斷時,IRQ 的個數為 224 個,否則為 16 個
當系統初始化時,核心呼叫 init_IRQ() 函式將 irq_desc 陣列中的 status 欄位設定為 IRQ_DISABLED
並且呼叫 set_intr_gate() 函式設定 CPU 的 Interrupt Gate,這個動作就是設定中斷發生時的處理函式

定義於 arch/i386/kernel/i8259.c

void __init init_IRQ(void)
{
  int i;

  /* all the set up before the call gates are initialised */
  pre_intr_init_hook();

  /*
   * Cover the whole vector space, no vector can escape
   * us. (some of these will be overridden and become
   * 'special' SMP interrupts)
   */
  for (i = 0; i < (NR_VECTORS - FIRST_EXTERNAL_VECTOR); i++) {
    int vector = FIRST_EXTERNAL_VECTOR + i;
    if (i >= NR_IRQS)
      break;
    if (vector != SYSCALL_VECTOR)
      set_intr_gate(vector, interrupt[i]);
  }

  /* setup after call gates are initialised (usually add in
   * the architecture specific gates)
   */
  intr_init_hook();

  /*
   * Set the clock to HZ Hz, we already have a valid
   * vector now:
   */
  setup_pit_timer();

  /*
   * External FPU? Set up irq13 if so, for
   * original braindamaged IBM FERR coupling.
   */
  if (boot_cpu_data.hard_math && !cpu_has_fpu)
    setup_irq(FPU_IRQ, &fpu_irq);

  irq_ctx_init(smp_processor_id());
}


void __init init_ISA_irqs (void)
{
  int i;

#ifdef CONFIG_X86_LOCAL_APIC
  init_bsp_APIC();
#endif
  init_8259A(0);

  for (i = 0; i < NR_IRQS; i++) {
    irq_desc[i].status = IRQ_DISABLED;
    irq_desc[i].action = NULL;
    irq_desc[i].depth = 1;

    if (i < 16) {
      /*
       * 16 old-style INTA-cycle interrupts:
       */
      irq_desc[i].handler = &i8259A_irq_type;
    } else {
      /*
       * 'high' PCI IRQs filled in on demand
       */
      irq_desc[i].handler = &no_irq_type;
    }
  }
}

2009年2月15日 星期日

Linux Interrupts and Exceptions - 1

Intel x86 處理器將中斷分成 同步的 Exception 以及 非同步的 Interrupt 兩類
並且給與 0-255 的識別號碼,稱之為 vector,在 Linux 核心則被稱為 IRQ Number

• Exceptions
由 CPU 產生的中斷,例如:執行指令時發生錯誤,或是使用者程式的系統呼叫
目前 x86 處理器定義了 20 個 Exception,即 IRQ Number 0-19,如下表所示:


• Interrupts
由其他硬體裝置或計時器發出的中斷,通常是將裝置的 IRQ Line 全部接到一顆
Programmable Interrupt Controller (PIC) 的控制晶片上,當裝置發出中斷時,由這顆 PIC 來通知 CPU
舊型的 PC 會使用兩個 Intel 8259 來串接成為中斷控制器,可處理 15 個硬體裝置的中斷要求
x86 處理器的 Interrupt 則是定義為 IRQ Number 32-255,如下圖所示:

對於多處理器的系統,則是使用新型的 Advanced Programmable Interrupt Controller (APIC)
來代替 8259 管理裝置的中斷,並具有單處理器系統的相容性

APIC 是由 Local APIC (LAPIC) 及 I/O APIC (IOAPIC) 兩部份所組成,LAPIC 位於 CPU 端,IOAPIC 位於南橋晶片上
多處理器系統的每個 CPU 會有一個 LAPIC,搭配一個連接硬體裝置的 IOAPIC,如下圖:



此外,x86 處理器使用 8 bytes 的 IDT descriptors 來描述中斷,分為三種型別:
1. Task Gate
2. Interrupt Gate
3. Trap Gate


Linux 則使用 256*8 = 2048 bytes 的空間,來儲存 CPU 全部 256 個中斷的 IDT 值
並且以 Interrupt Gate 來處理 Interrupt 的中斷,以 Trap Gate 來處理 Exception 的中斷,如下:

struct desc_struct { unsigned long a,b;};

定義於 include/asm-i386/processor.h


struct desc_struct idt_table[256] __attribute__((__section__(".data.idt"))) = { {0, 0}, };

定義於 arch/i386/kernel/traps.c

2009年1月11日 星期日

單車遊記 - 三坑自行車道

在2008年的最後一個假日來到 三坑自行車道 探路,是一條由大溪通往龍潭三坑老街的自行車道
根據路線圖的指示,可以由大溪橋出發走主線到三坑老街,再走其他支線繞一圈回到大溪橋


這條自行車道頗具特色:從大溪橋開始會經過韭菜田,到了三坑便是老街的風景
走另一條支線的話,可以看到石門大圳的水渠道。此外,大漢溪旁的自然生態公園相當別緻
是一個不錯的休息點,走上大漢溪的河堤還可以看到遠方的石門水庫呢~




騎完三坑自行車道後,亦可順道一遊大溪老街
全部的照片在此:
http://picasaweb.google.com/yichung.shen/20081228

2008年12月17日 星期三

2008 日本關西之五

[伏見稻荷大社]
第七天 因為是最後一天,隊友們都想睡到自然醒,所以還是獨自行動
首先,從京都車站搭乘JR到 伏見稻荷大社 一遊,神社就在JR稻荷站的對面
這個神社可是日本全國各地的稻荷神社之首,祭祠稻荷神,使者是狐貍
還有超多的鳥居,由來此許願的人所捐款豎立的,形成 "千本鳥居" 的特色
JR奈良線 稻荷站

[宇治 平等院]
接著繼續搭上JR前往宇治,而宇治可去的地方很多,也是源氏物語的主要舞台
但是今天只有半天的時間,所以直接去看最有名的 平等院,又稱為 鳳凰堂
在安平時代就建造了平等院,10元日幣背面的圖案就是這個了
JR奈良線 宇治站

到了宇治記得要吃吃有加抹茶粉的 抹茶霜淇淋喔~

由京都去關西空港時,使用 KANSAI THRU PASS,從京都車站搭近畿日本鐵道的宇治線
-> 大和西大寺站 轉 奈良線 -> 難波站 轉 南海電鐵的南海空港線

- 完 -

2008年12月14日 星期日

2008 日本關西之四

[京都市區]
第六天雨停了,可以租自轉車,今天要騎車來逛京都市區
KCTP 有提供自轉車出租,而且設有幾個不同的服務處可以取車、還車
這次是在錦市場附近的 Co-op Inn Kyoto Hotel 租的,離我們住的旅館也最近

以下是今天的路線:

順著河原町通往北走,先到達 下鴨神社 以及更北一點的 上賀茂神社,兩個都是京都最古老的神社


之後轉到堀川通往南走,會先到 晴明神社,就在西陣織會館旁邊
這是祭祠陰陽師安倍晴明的神社,陰陽師迷必來的地方,五芒星的繪馬是其特色


再往南走是 東、西本願寺,和前述所參觀的神社不同的是,本願寺是佛教的寺廟
在此可體驗到傳統的日本佛教及有別於神社的寺廟建築


本願寺之後便去附近的 涉成園,這是一個優美的日式庭園,屬於東本願寺的別院

最後一站是位於七條通上的 三十三間堂,其名稱是因為正殿內有33根立柱隔開的間隔
三十三間堂最有看頭的是供奉著1001座千手觀世音菩薩像,非常壯觀,不過裏面是禁止攝影的喔

回程則走川端通,沿著鴨川北上,騎車回到祗園及四條河原町

2008年12月9日 星期二

2008 日本關西之三

[時代祭]
第四天(10/22)在京都有兩個祭典,中午開始的是京都每年三大祭之一的 時代祭,晚上則是鞍馬的火祭


於是早上就先從大阪移動到京都,並且在四條河原町本店享用傳說中的 "名代豬排",接著時代祭就開始了
時代祭的遊行路線是從京都御所至平安神宮,我們就在河原町通的市役所附近觀看


整個祭典是以時光倒流的方式呈現,由明治維新開始往前演示日本歷史上的各個時代
很可惜的是天氣不好,遊行開始沒多久就下起雨了,而且連下三天的雨...
地下鐵東西線 京都市役所前站

[鞍馬火祭]
火祭是在鞍馬的由岐神社舉行,由 18:00開始到 24:00才結束,整個祭典活動的時間相當長
但是遊客超多,為了避開回程時可能遇到的人潮,火祭看了兩個小時我們就離開了
這時都是在進行點火把的儀式,其實祭典的主要部份還沒開始吧~
叡山電車鞍馬線 鞍馬站

[天橋立]
第五天大家安排京都的市區觀光,我就去有日本三景之稱的 "宮津天橋立"
天橋立位於京都府西北方的宮津灣內,其特色是有一條狹長的沙洲,而且沙洲可是長滿了松樹
沙洲的兩邊都有通往小山丘的纜車,在山丘上彎腰由胯下望去時,沙洲就好像天上的一座橋
這次是由 JR天橋立站開始步行走過 "天橋" 到對岸的傘松公園,再坐船回來


不過去天橋立要花不少時間,最簡單的交通方式是搭 JR從京都車站至天橋立站的直達車
詳情可見官方網站 http://www.amanohashidate.jp/index.html

2008年11月23日 星期日

2008 日本關西之二

[四天王寺]
第一天從機場出關、坐電車到大阪市,再找到旅館做完Check In,時間已經是下午四點了
決定去位於旅館同一區的四天王寺,但是最後進場時間是15:30,晚了一步,殘念~
地下鐵谷町線 四天王寺前夕陽之丘站 4號出口


[姬路城]
第二天去姬路城,位於兵庫縣姬路市的城堡。從大阪市區過去的話,距離稍遠,但是很值得去
因為這個城存保得很好,加上外觀典雅,城的內部也有看頭,同時也是世界遺產喔
這裏如果慢慢逛,可以待一整個下午
山陽電鐵本線 山陽姬路站


[心齋橋+道頓堀]
心齋橋是一條長長的商店街,是瞎拼、敗家的好地方。而道頓堀是美食區,就在心齋橋的旁邊
在這裏當然就是盡情的買東西、吃東西啦~
地下鐵御堂筋線 心齋橋筋站

[和歌山城]
第三天所有人都去環球影城了,因為之前去過一次,所以今天決定去和歌山城以及拜訪貓咪站長
和歌山地區有不少可去之處,但礙於交通及時間問題,上午只去了和歌山城
其實昨天才去過姬路城,相較之下,和歌山城就遜色不少...
JR阪和線 和歌山站 再轉公車


[貴志 貓咪站長]
下午則乘坐 JR貴志川線,去看看日本鐵道的超級站長 たま
這條鐵路支線原本沒什麼人氣,而終點站 貴志 是一個無人車站,倒是車站的商店老闆
養了一隻三色貓 たま,鐵路公司為了振興該線的運輸量,想到任命 たま 擔任貴志站站長的點子
果然貓咪站長馬上爆紅,後來還升官成為 "超級站長" 了呢!!


貴志川線官網 http://www.wakayama-dentetsu.co.jp/