教程弹性搜索 - 导入 CSV 文件 [ 一步一步 ]

您想了解如何将 CSV 文件导入弹性搜索吗？在本教程中，我们将向您展示如何使用 Logstash 或 Kibana 将 CSV 文件导入弹性搜索服务器。

• 乌本图 20
• 乌本图 19
• 乌本图 18
• 弹性搜索 7.6.2
• 日志 7.6.2

在我们的示例中，ElastiSearch 服务器 IP 地址为 192.168.100.9。

在我们的示例中，我们将安装 Logstash 并导入 CSV 文件。

弹性搜索相关教程：

在此页上，我们提供快速访问与弹性搜索安装相关的教程列表。

教程日志 – 安装

使用 apt-get 安装所需的软件包。

Copy to Clipboard

查找 Java 安装目录。

Copy to Clipboard

在我们的示例中，Java 安装在以下目录上： /usr/lib/jvm/java-11-openjdk-amd64

创建文件以配置所需的 Java 变量。

Copy to Clipboard

这是文件内容。

Copy to Clipboard

重新启动计算机。

Copy to Clipboard

验证系统是否自动创建了名为：JAVA_HOME的环境变量。

Copy to Clipboard

下面是命令输出。

Copy to Clipboard

下载并安装弹性搜索存储库密钥。

Copy to Clipboard

将官方弹性搜索存储库添加到 APT 数据库

Copy to Clipboard

更新 APT-GET 数据库并安装 Logstash 包。

Copy to Clipboard

将 Logstash 服务配置为自动启动。

Copy to Clipboard

祝贺！您已完成 Ubuntu Linux 上的 Logstash 安装。

教程弹性搜索 – 导入 CSV 文件

创建 CSV 文件。

Copy to Clipboard

这是我们的文件内容。

Copy to Clipboard

在我们的示例中，我们有一个 CSV 文件，显示比特币的每日价格。

创建 Logtash 配置文件。

Copy to Clipboard

这是我们的文件内容。

Copy to Clipboard

停止 Logstash 服务。

Copy to Clipboard

使用新的配置文件对 Logstash 服务进行星形。

Copy to Clipboard

下面是命令输出。

Copy to Clipboard

{
    "market-price" => 0.0,
        "@version" => "1",
        "datetime" => "datetime",
            "tags" => [
        [0] "_dateparsefailure"
    ],
            "path" => "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
         "message" => "datetime,market-price",
      "@timestamp" => 2020-05-11T01:03:13.387Z,
            "host" => "elasticsearch-01.local"
}
{
    "market-price" => 5657.14,
        "@version" => "1",
        "datetime" => "2019-05-04 00:00:00",
            "path" => "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
         "message" => "2019-05-04 00:00:00,5657.14",
      "@timestamp" => 2019-05-04T03:00:00.000Z,
            "host" => "elasticsearch-01.local"
}
{
    "market-price" => 5771.08,
        "@version" => "1",
        "datetime" => "2019-05-05 00:00:00",
            "path" => "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
         "message" => "2019-05-05 00:00:00,5771.08",
      "@timestamp" => 2019-05-05T03:00:00.000Z,
            "host" => "elasticsearch-01.local"
}
{
    "market-price" => 5717.66,
        "@version" => "1",
        "datetime" => "2019-05-06 00:00:00",
            "path" => "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
         "message" => "2019-05-06 00:00:00,5717.66",
      "@timestamp" => 2019-05-06T03:00:00.000Z,
            "host" => "elasticsearch-01.local"
}
{
    "market-price" => 5684.47,
        "@version" => "1",
        "datetime" => "2019-05-07 00:00:00",
            "path" => "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
         "message" => "2019-05-07 00:00:00,5684.47",
      "@timestamp" => 2019-05-07T03:00:00.000Z,
            "host" => "elasticsearch-01.local"
}

在我们的示例中，我们使用 Logstash 将包含 2 个字段的 CSV 文件导入弹性搜索。

在我们的示例中，我们将包含比特币价格的字段转换为数字浮点类型。

在我们的示例中，我们将名为 DATETIME 的字段设置为默认时间戳。

验证索引是否是在弹性搜索服务器上创建的。

Copy to Clipboard

下面是命令输出。

Copy to Clipboard

验证比特币价格是否映射为数值。

Copy to Clipboard

下面是命令输出。

Copy to Clipboard

{
  "bitcoin" : {
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "datetime" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "host" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "market-price" : {
          "type" : "float"
        },
        "message" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "path" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "tags" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

在我们的示例中，名为 MARKET-PRICE 的字段设置为浮点数。

列出名为比特币的索引的内容。

Copy to Clipboard

下面是命令输出。

Copy to Clipboard

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 11,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "18eP1nEBeI5DkG2uuA6v",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "market-price",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "Timestamp,market-price",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "Timestamp",
          "@timestamp" : "2020-05-02T18:04:33.224Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "2MeP1nEBeI5DkG2uuA6v",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5657.14",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-04 00:00:00,5657.14",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-04 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.290Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "2ceP1nEBeI5DkG2uuA6v",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5771.08",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-05 00:00:00,5771.08",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-05 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.292Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "2seP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5717.66",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-06 00:00:00,5717.66",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-06 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.298Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "28eP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5684.47",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-07 00:00:00,5684.47",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-07 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.298Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "3MeP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5755.72",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-08 00:00:00,5755.72",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-08 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.299Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "3ceP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "5936.72",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-09 00:00:00,5936.72",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-09 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.301Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "3seP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "6146.91",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-10 00:00:00,6146.91",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-10 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.301Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "38eP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "6348.02",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-11 00:00:00,6348.02",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-11 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.302Z"
        }
      },
      {
        "_index" : "bitcoin",
        "_type" : "_doc",
        "_id" : "4MeP1nEBeI5DkG2uuA6w",
        "_score" : 1.0,
        "_source" : {
          "market-price" : "7248.31",
          "host" : "elasticsearch-01.local",
          "@version" : "1",
          "message" : "2019-05-12 00:00:00,7248.31",
          "path" : "https://d1ny9casiyy5u5.cloudfront.net/downloads/bitcoin.csv",
          "Timestamp" : "2019-05-12 00:00:00",
          "@timestamp" : "2020-05-02T18:04:33.302Z"
        }
      }
    ]
  }
}

祝贺！将 CSV 文件导入弹性搜索。

弹性搜索 – 导入 CSV 文件

弹性搜索 – 导入 CSV 文件

弹性搜索相关教程：

教程日志 – 安装

教程弹性搜索 – 导入 CSV 文件

Related Posts