如何将数组插入数据库？

Question

在我之前的问题中,很多用户希望我给玩具提供更多数据.所以我开始导出所有数据并用Python处理它,但后来我意识到:我在哪里留下所有这些数据？

好吧我决定最好将它们放在数据库中,所以至少我不必每次都解析原始文件.但由于我对数据库一无所知,因此结果令人困惑.我尝试了一些教程来创建一个sqlite数据库,添加一个表和字段并尝试插入我的numpy.arrays,但它无法让它工作.

通常我的每条狗的结果看起来像这样:

所以我有35只不同的狗,每只狗有24个测量值.每个测量本身都有未知数量的联系人.每个测量由3D阵列(整个板的248帧[255x63])和2D阵列(板的每个传感器的最大值[255x63])组成.在数据库中存储一个值不是问题,但在那里获取我的2D数组似乎不起作用.

所以我的问题是如何在数据库中订购并将数组插入其中？

Answer 1

你可能想要从dogs包含每只狗的所有扁平(非阵列)数据的表开始,每只狗都有一个,比如姓名,性别和年龄:

CREATE TABLE `dogs` (
  `id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
  `name` VARCHAR(64),
  `age` INT UNSIGNED,
  `sex` ENUM('Male','Female')
);

从那里,每只狗"有很多"测量,所以你需要一个dog_mesaurements表来存储24个测量值:

CREATE TABLE `dog_measurements` (
  `id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
  `dog_id` INT UNSIGNED NOT NULL,
  `paw` ENUM ('Front Left','Front Right','Rear Left','Rear Right'),
  `taken_at` DATETIME NOT NULL
);

那么每当你进行测量时,你INSERT INTO dog_measurements (dog_id,taken_at) VALUES (*?*, NOW());在哪里*？*是dogs表格中的狗的ID .

然后,您需要表格来存储每个测量的实际帧,例如:

CREATE TABLE `dog_measurement_data` (
  `id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
  `dog_measurement_id` INT UNSIGNED NOT NULL,
  `frame` INT UNSIGNED,
  `sensor_row` INT UNSIGNED,
  `sensor_col` INT UNSIGNED,
  `value` NUMBER
);

这样,对于250帧中的每一帧,您循环遍历63个传感器中的每一个,并将具有帧编号的该传感器的值存储到数据库中:

INSERT INTO `dog_measurement_data` (`dog_measurement_id`,`frame`,`sensor_row`,`sensor_col`,`value`) VALUES
(*measurement_id?*, *frame_number?*, *sensor_row?*, *sensor_col?*, *value?*)

显然要替换measurement_id？,frame_number？,sensor_number？,价值？有实际价值:-)

所以基本上,每个dog_measurement_data都是给定帧的单个传感器值.这样,为了获得所有给定帧的所有传感器值,您将:

SELECT `sensor_row`,sensor_col`,`value` FROM `dog_measurement_data`
WHERE `dog_measurement_id`=*some measurement id* AND `frame`=*some frame number*
ORDER BY `sensor_row`,`sensor_col`

这将为您提供该帧的所有行和列.

Answer 2

Django有一个库,用于将所有数据库工作封装到Python类中,因此在必须做一些非常聪明的事情之前,您不必乱用原始SQL.尽管Django是Web应用程序的框架,但您可以单独使用数据库ORM.

Josh的模型在使用Django的Python中看起来像这样:

from django.db import models

class Dog(models.Model):
    # Might want to look at storing birthday instead of age.
    # If you track age, you probably need another field telling
    # you when in the year age goes up by 1... and at that point,
    # you're really storing a birthday.
    name = models.CharField(max_length=64)
    age = models.IntegerField()
    genders = [
        ('M', 'Male'),
        ('F', 'Female'),
    ]
    gender = models.CharField(max_length=1, choices=genders)

class Measurement(models.Model):
    dog = models.ForeignKey(Dog, related_name="measurements")
    paws = [
        ('FL', 'Front Left'),
        ('FR', 'Front Right'),
        ('RL', 'Rear Left'),
        ('RR', 'Rear Right'),
    ]
    paw = models.CharField(max_length=2, choices=paws)
    taken_at = models.DateTimeField(default=date, auto_now_add=True)

class Measurement_Point(models.Model):
    measurement = models.ForeignKey(Measurement, related_name="data_points")
    frame = models.IntegerField()
    sensor_row = models.PositiveIntegerField()
    sensor_col = models.PositiveIntegerField()
    value = models.FloatField()

    class Meta:
        ordering = ['frame', 'sensor_row', 'sensor_col']

该id字段自动创建.

然后你可以做以下事情:

dog = Dog()
dog.name = "Pochi"
dog.age = 3
dog.gender = 'M'
# dog.gender will return 'M', and dog.get_gender_display() will return 'Male'
dog.save()

# Or, written another way:
dog = Dog.objects.create(name="Fido", age=3, sex='M')

要进行测量:

measurement = dog.measurements.create(paw='FL')
for frame in range(248):
    for row in range(255):
        for col in range(63):
            measurement.data_points.create(frame=frame, sensor_row=row, 
                sensor_col=col, value=myData[frame][row][col])

最后,获得一个框架:

# For the sake of argument, assuming the dogs have unique names.
# If not, you'll need some more fields in the Dog model to disambiguate.
dog = Dog.objects.get(name="Pochi", sex='M')
# For example, grab the latest measurement...
measurement = dog.measurements.all().order_by('-taken_at')[0]
# `theFrameNumber` has to be set somewhere...
theFrame = measurement.filter(frame=theFrameNumber).values_list('value')

注意:这将返回元组列表(例如[(1.5,), (1.8,), ... ]),因为values_list()可以一次检索多个字段.我不熟悉NumPy,但我想它有一个类似于Matlab reshape函数的函数,用于将向量重新映射到矩阵.