小编Jon*_*ito的帖子

尚不支持pandas + pyodbc ODBC SQL类型-150

我知道有很多话题,但是我认为这很具体。我得到用于审计目的的当前代码:

import pandas as pd
import pyodbc

query = """
--Top 50 high total CPU Queries
SELECT TOP 50
'High CPU Queries' as Type,
serverproperty('machinename') as 'Server Name',
isnull(serverproperty('instancename'),serverproperty('machinename')) as 'Instance Name',
        COALESCE(DB_NAME(qt.dbid),
        DB_NAME(CAST(pa.value as int)), 
        'Resource') AS DBNAME,
    qs.execution_count as [Execution Count],
    qs.total_worker_time/1000 as [Total CPU Time],
    (qs.total_worker_time/1000)/qs.execution_count as [Avg CPU Time],
    qs.total_elapsed_time/1000 as [Total Duration],
    (qs.total_elapsed_time/1000)/qs.execution_count as [Avg Duration],
    qs.total_physical_reads as [Total Physical Reads],
    qs.total_physical_reads/qs.execution_count as [Avg Physical Reads],
    qs.total_logical_reads as [Total Logical Reads],
    qs.total_logical_reads/qs.execution_count as [Avg Logical …
Run Code Online (Sandbox Code Playgroud)

sql-server odbc pyodbc pandas

3
推荐指数
2
解决办法
1846
查看次数

在 athena 中创建 terraform 数据库

我正在尝试使用 terraform 创建一个数据库,这对于一个糟糕的查询来说似乎非常复杂......

请问你能帮帮我吗?

我已经尝试过 null_resource 与 local-exec 和数据“外部”Python ...我想我看错了方式

ex 在 terraform 0.12 中不起作用

resource "null_resource" "create-endpoint" {
  provisioner "local-exec" {
  query = <<EOF
{
CREATE EXTERNAL TABLE `dashboard_loading_time`(
  `timestamp_iso` string, 
  `app_identification` struct<service:string,app_name:string,app_type:string,stage:string>, 
  `user` struct<api_gateway_key:struct<id:string,name:string>,mashery_key:struct<id:string,name:string>,employee:struct<id:string,name:string>>, 
  `action` struct<action_type:string,path:string>, 
  `result` struct<status:string,http_status:string,response:struct<response:string>>)
PARTITIONED BY ( 
  `year` int)
ROW FORMAT SERDE 
  'org.openx.data.jsonserde.JsonSerDe' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  's3://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/dev'
}
EOF
    command = "aws athena start-query-execution --query-string "query""
  }
}
Run Code Online (Sandbox Code Playgroud)

我想找到使用 terraform 执行此操作的最简单方法。

database creation terraform aws-glue

1
推荐指数
1
解决办法
5092
查看次数

Terraform 粘合作业无法正确创建

我正在使用 terraform,但没有获得正确的参数来创建我的粘合作业。由于我不是 terraform 专业人士(我开始),我想知道它是如何工作的。

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/glue_job#glue_version

我没有使用这些参数对我的胶水作业资源进行良好的更新:

resource "aws_glue_job" "job_name" {
  name     = "job_name"
  description  = "job-desc"
  role_arn = "${aws_iam_role.service-name.arn}"
  max_capacity  = 2
  max_retries  = 1
  timeout      = 60

  command {
    script_location = "s3://my_bucket"
    python_version  = "3"
  }

  default_arguments = {    
    "--job-language"    = "python"
    "--ENV"             = "env"
    "--spark-event-logs-path" = "s3://my_bucket"
    "--job-bookmark-option" = "job-bookmark-enable"
    "--glue_version" = "2.0"
    "--worker_type" = "G.1X"
    "--enable-spark-ui" = "true"
  }

  execution_property {
    max_concurrent_runs = 1
  }
}
Run Code Online (Sandbox Code Playgroud)

我不知道这些参数放在哪里以及如何放置。请你帮助我好吗 ?

    "--glue_version" = "2.0"
    "--worker_type" = "G.1X"
Run Code Online (Sandbox Code Playgroud)

问候。

terraform aws-glue

1
推荐指数
1
解决办法
4998
查看次数

标签 统计

aws-glue ×2

terraform ×2

creation ×1

database ×1

odbc ×1

pandas ×1

pyodbc ×1

sql-server ×1