使用Cpuset将内核模块隔离到特定的核心

Question

从用户空间我们可以使用cpuset实际隔离系统中的特定核心,并只执行一个特定的核心进程.

我正在尝试使用内核模块做同样的事情.所以我希望模块在一个孤立的核心中执行.换句话说:我如何cpuset在内核模块中使用？*

在我的内核模块中使用linux/cpuset.h不起作用.所以,我有一个这样的模块:

#include <linux/module.h>
#include <linux/cpuset.h>

...
#ifdef CONFIG_CPUSETS
    printk(KERN_INFO, "cpusets is enabled!");
#endif
cpuset_init(); // this function is declared in cpuset.h
...

尝试加载此模块时,我收到dmesg以下消息cpusets is enabled!.但我也收到了这条消息Unknown symbol cpu_init (err 0).

类似地,我尝试使用sched_setaffinityfrom linux/sched.h来将所有正在运行的procceses移动到特定的核心,然后将我的模块运行到一个隔离的核心.我得到了相同的错误消息:Unknown symbol sched_setaffinity (err 0).我想我得到了"未知符号",因为这些函数EXPORT_SYMBOL在内核中没有.所以我去尝试调用sys_sched_setaffinity 系统调用(基于这个问题),但又得到了这个消息:Unknown symbol sys_sched_setaffinity (err 0)!

此外,我不是在寻找一个使用的解决方案isolcpus,它在启动时设置.我想加载模块,然后发生隔离.

• (更确切地说,我希望它的内核线程在隔离的内核中执行.我知道我可以使用affinity将线程绑定到特定的内核,但这并不能保证内核会被运行在其上的其他进程隔离. )

Answer 1

所以我希望模块在一个孤立的核心中执行.

和

实际上是在我们的系统中隔离一个特定的核心,并只对该核心执行一个特定的过程

这是一个使用内核3.16在Debian盒子上编译和测试的工作源代码.我将描述如何首先加载和卸载以及传递的参数意味着什么.

所有来源都可以在github上找到...

https://github.com/harryjackson/doc/tree/master/linux/kernel/toy/toy

构建并加载模块......

make
insmod toy param_cpu_id=2

卸载模块使用

rmmod toy

我没有使用modprobe,因为它需要一些配置等.我们传递给toy内核模块的参数是我们想要隔离的CPU.除非它们在该CPU上执行,否则所有被调用的设备操作都不会运行.

加载模块后,您可以在此处找到它

/dev/toy

简单的操作就像

cat /dev/toy

创建内核模块捕获的事件并生成一些输出.您可以使用查看输出dmesg.

源代码...

#include <linux/module.h>
#include <linux/fs.h>
#include <linux/miscdevice.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Harry");
MODULE_DESCRIPTION("toy kernel module");
MODULE_VERSION("0.1"); 
#define  DEVICE_NAME "toy"
#define  CLASS_NAME  "toy"

static int    param_cpu_id;
module_param(param_cpu_id    , int, (S_IRUSR | S_IRGRP | S_IROTH));
MODULE_PARM_DESC(param_cpu_id, "CPU ID that operations run on");

//static void    bar(void *arg);
//static void    foo(void *cpu);
static int     toy_open(   struct inode *inodep, struct file *fp);
static ssize_t toy_read(   struct file *fp     , char *buffer, size_t len, loff_t * offset);
static ssize_t toy_write(  struct file *fp     , const char *buffer, size_t len, loff_t *);
static int     toy_release(struct inode *inodep, struct file *fp);

static struct file_operations toy_fops = {
  .owner = THIS_MODULE,
  .open = toy_open,
  .read = toy_read,
  .write = toy_write,
  .release = toy_release,
};

static struct miscdevice toy_device = {
  .minor = MISC_DYNAMIC_MINOR,
  .name = "toy",
  .fops = &toy_fops
};

//static int CPU_IDS[64] = {0};
static int toy_open(struct inode *inodep, struct file *filep) {
  int this_cpu = get_cpu();
  printk(KERN_INFO "open: called on CPU:%d\n", this_cpu);
  if(this_cpu == param_cpu_id) {
    printk(KERN_INFO "open: is on requested CPU: %d\n", smp_processor_id());
  }
  else {
    printk(KERN_INFO "open: not on requested CPU:%d\n", smp_processor_id());
  }
  put_cpu();
  return 0;
}
static ssize_t toy_read(struct file *filep, char *buffer, size_t len, loff_t *offset){
  int this_cpu = get_cpu();
  printk(KERN_INFO "read: called on CPU:%d\n", this_cpu);
  if(this_cpu == param_cpu_id) {
    printk(KERN_INFO "read: is on requested CPU: %d\n", smp_processor_id());
  }
  else {
    printk(KERN_INFO "read: not on requested CPU:%d\n", smp_processor_id());
  }
  put_cpu();
  return 0;
}
static ssize_t toy_write(struct file *filep, const char *buffer, size_t len, loff_t *offset){
  int this_cpu = get_cpu();
  printk(KERN_INFO "write called on CPU:%d\n", this_cpu);
  if(this_cpu == param_cpu_id) {
    printk(KERN_INFO "write: is on requested CPU: %d\n", smp_processor_id());
  }
  else {
    printk(KERN_INFO "write: not on requested CPU:%d\n", smp_processor_id());
  }
  put_cpu();
  return 0;
}
static int toy_release(struct inode *inodep, struct file *filep){
  int this_cpu = get_cpu();
  printk(KERN_INFO "release called on CPU:%d\n", this_cpu);
  if(this_cpu == param_cpu_id) {
    printk(KERN_INFO "release: is on requested CPU: %d\n", smp_processor_id());
  }
  else {
    printk(KERN_INFO "release: not on requested CPU:%d\n", smp_processor_id());
  }
  put_cpu();
  return 0;
}

static int __init toy_init(void) {
  int cpu_id;
  if(param_cpu_id < 0 || param_cpu_id > 4) {
    printk(KERN_INFO "toy: unable to load module without cpu parameter\n");
    return -1;
  }
  printk(KERN_INFO "toy: loading to device driver, param_cpu_id: %d\n", param_cpu_id);
  //preempt_disable(); // See notes below
  cpu_id = get_cpu();
  printk(KERN_INFO "toy init called and running on CPU: %d\n", cpu_id);
  misc_register(&toy_device);
  //preempt_enable(); // See notes below
  put_cpu();
  //smp_call_function_single(1,foo,(void *)(uintptr_t) 1,1);
  return 0;
}

static void __exit toy_exit(void) {
    misc_deregister(&toy_device);
    printk(KERN_INFO "toy exit called\n");
}

module_init(toy_init);
module_exit(toy_exit); 

上面的代码包含您要求的两种方法,即隔离CPU和在init隔离内核上运行.

在init上get_cpu禁用抢占,即在它之后的任何内容都不会被内核抢占并将在一个核心上运行.注意,这是使用3.16完成的内核,你的里程可能因内核版本而异,但我认为这些API已经存在了很长时间

这是Makefile ......

obj-m += toy.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

笔记.get_cpu声明linux/smp.h为

#define get_cpu()   ({ preempt_disable(); smp_processor_id(); })
#define put_cpu()   preempt_enable()

所以你在打电话preempt_disable之前实际上并不需要打电话get_cpu.get_cpu调用是以下一系列调用的包装器...

preempt_count_inc();
barrier();

和put_cpu真的这样做......

barrier();
if (unlikely(preempt_count_dec_and_test())) {
  __preempt_schedule();
}   

您可以使用上述内容随心所欲.几乎所有这些都来自以下来源..

Google for ... smp_call_function_single

Linux内核开发,Robert Love的书.

http://derekmolloy.ie/writing-a-linux-kernel-module-part-2-a-character-device/

https://github.com/vsinitsyn/reverse/blob/master/reverse.c