当前位置: 首页 > news >正文

windows 11 配置 kafka 使用SASL SCRAM-SHA-256 认证

1. 下载安装apache-zookeeper-3.9.2

配置 \conf\zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=D:/server/data/zookeeper
dataLogDir=D:/server/data/zookeeperLog
# admin管理端端口
#admin.serverPort=8887# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpHost=0.0.0.0
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true
2. 启动 zkServer.cmd

3. 下载并重命名 kafka , kafka2.12.3.8.0

4. 编辑配置 \config\server.properties 以使用 SCRAM-SHA-256
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#
# This configuration file is intended for use in ZK-based mode, where Apache ZooKeeper is required.
# See kafka.server.KafkaConfig for additional details and defaults
############################## Server Basics ############################## The id of the broker. This must be set to a unique integer for each broker.
broker.id=0############################# Socket Server Settings ############################## The address the socket server listens on. If not configured, the host name will be equal to the value of
# java.net.InetAddress.getCanonicalHostName(), with PLAINTEXT listener name, and port 9092.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092
# 启用 SASL 机制
sasl.enabled.mechanisms=SCRAM-SHA-256# 配置 SASL 认证方式
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
security.inter.broker.protocol=SASL_PLAINTEXT# 启用用户认证的 Kafka 认证控制
allow.everyone.if.no.acl.found=false
# 配置 Kafka 用户的 JAAS 配置
listener.name.sasl_plaintext.scram-sha-256.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \username="kafka" \password="kafkaAdmin#20240304";# 如果使用 SSL, 配置 SSL 文件路径(可选)
# ssl.keystore.location=/path/to/keystore.jks
# ssl.keystore.password=your_keystore_password
# ssl.key.password=your_key_password# 设置Kafka 端口
listeners=SASL_PLAINTEXT://127.0.0.1:9092
advertised.listeners=SASL_PLAINTEXT://127.0.0.1:9092# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
#advertised.listeners=PLAINTEXT://your.host.name:9092# Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details
#listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL# The number of threads that the server uses for receiving requests from the network and sending responses to the network
num.network.threads=3# The number of threads that the server uses for processing requests, which may include disk I/O
num.io.threads=8# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=102400# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=102400# The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600############################# Log Basics ############################## A comma separated list of directories under which to store log files
# 设置Kafka日志
log.dirs=D:/server/data/kafka-logs# The default number of log partitions per topic. More partitions allow greater
# parallelism for consumption, but this will also result in more files across
# the brokers.
num.partitions=1# The number of threads per data directory to be used for log recovery at startup and flushing at shutdown.
# This value is recommended to be increased for installations with data dirs located in RAID array.
num.recovery.threads.per.data.dir=1############################# Internal Topic Settings  #############################
# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"
# For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3.
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1############################# Log Flush Policy ############################## Messages are immediately written to the filesystem but by default we only fsync() to sync
# the OS cache lazily. The following configurations control the flush of data to disk.
# There are a few important trade-offs here:
#    1. Durability: Unflushed data may be lost if you are not using replication.
#    2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush.
#    3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to excessive seeks.
# The settings below allow one to configure the flush policy to flush data after a period of time or
# every N messages (or both). This can be done globally and overridden on a per-topic basis.# The number of messages to accept before forcing a flush of data to disk
#log.flush.interval.messages=10000# The maximum amount of time a message can sit in a log before we force a flush
#log.flush.interval.ms=1000############################# Log Retention Policy ############################## The following configurations control the disposal of log segments. The policy can
# be set to delete segments after a period of time, or after a given size has accumulated.
# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
# from the end of the log.# The minimum age of a log file to be eligible for deletion due to age
log.retention.hours=168# A size-based retention policy for logs. Segments are pruned from the log unless the remaining
# segments drop below log.retention.bytes. Functions independently of log.retention.hours.
#log.retention.bytes=1073741824# The maximum size of a log segment file. When this size is reached a new log segment will be created.
#log.segment.bytes=1073741824# The interval at which log segments are checked to see if they can be deleted according
# to the retention policies
log.retention.check.interval.ms=300000############################# Zookeeper ############################## Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
#设置ZK地址
zookeeper.connect=localhost:2181# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=18000############################# Group Coordinator Settings ############################## The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance.
# The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms.
# The default value for this is 3 seconds.
# We override this to 0 here as it makes for a better out-of-the-box experience for development and testing.
# However, in production environments the default value of 3 seconds is more suitable as this will help to avoid unnecessary, and potentially expensive, rebalances during application startup.
group.initial.rebalance.delay.ms=0

配置 JAAS 文件 \kafka2.12.3.8.0\config\kafka_server_jaas.conf

KafkaServer {org.apache.kafka.common.security.scram.ScramLoginModule requiredusername="kafka"password="kafkaAdmin#20240304";
};Client {org.apache.kafka.common.security.scram.ScramLoginModule requiredusername="kafka"password="kafkaAdmin#20240304";
};

设置环境变量 (optional)

set KAFKA_OPTS=-Djava.security.auth.login.config=D:/server/kafka2.12.3.8.0/config/kafka_server_jaas.conf

为 kafka 用户创建 SCRAM-SHA-256 凭据

\kafka2.12.3.8.0\bin\windows\kafka-configs --zookeeper localhost:2181 --alter --add-config SCRAM-SHA-256=[iterations=4096,password=kafkaAdmin#20240304],SCRAM-SHA-512=[password=kafkaAdmin#20240304] --entity-type users --entity-name kafka

配置 Kafka 客户端 \kafka2.12.3.8.0\config\client.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="kafka" \
  password="kafkaAdmin#20240304";

5. 启动kafka kafka-server-start.bat ..\..\config\server.properties
6. 测试配置

新建主题

kafka-topics --create --topic test-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1 --command-config ../../config/client.properties
 

新建生产者 并输入数据

kafka-console-producer --topic test-topic --bootstrap-server localhost:9092 --producer.config  ../../config/client.properties
 

新建消费者以显示生产者数据

kafka-console-consumer --topic test-topic --bootstrap-server localhost:9092 --from-beginning --consumer.config ../../config/client.properties
 

测试成功

7. 安装配置kafka UI 消息查看工具

下载git 代码打包生产 kafdrop-4.0.3-SNAPSHOT.jar (git 跳过 test), 路径D:\soft\kafka

配置UI客户端连接文件 kafka.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="kafka" password="kafkaAdmin#20240304";

命令行启动UI

java --add-opens=java.base/sun.nio.ch=ALL-UNNAMED -jar kafdrop-4.0.3-SNAPSHOT.jar  --kafka.brokerConnect=127.0.0.1:9092 --kafka.properties=kafka.properties

或者指定控制台端口

java --add-opens=java.base/sun.nio.ch=ALL-UNNAMED -jar kafdrop-4.0.3-SNAPSHOT.jar  --kafka.brokerConnect=localhost:9092 --server.port=9999 --management.server.port=9999 --kafka.properties=kafka.properties
 

登录控制台查看 URL: http://localhost:9000/ 可查看先前 topic: test-topic

windows11 WSL 配置后只能WSL下使用,主机 SCRAM-SHA-256 配置访问不成功,故放弃.

看来WSL只能简单使用一些工具,多安全网络相关配置不适用

相关文章:

windows 11 配置 kafka 使用SASL SCRAM-SHA-256 认证

1. 下载安装apache-zookeeper-3.9.2 配置 \conf\zoo.cfg # The number of milliseconds of each tick tickTime2000 # The number of ticks that the initial # synchronization phase can take initLimit10 # The number of ticks that can pass between # sending a requ…...

Elasticsearch —— ES 环境搭建、概念、基本操作、文档操作、SpringBoot继承ES

文章中会用到的文件,如果官网下不了可以在这下 链接: https://pan.baidu.com/s/1SeRdqLo0E0CmaVJdoZs_nQ?pwdxr76 提取码: xr76 一、 ES 环境搭建 注:环境搭建过程中的命令窗口不能关闭,关闭了服务就会关闭(除了修改设置后重启的…...

ElSelect 组件的 onChange 和 onInput 事件的区别

偶然遇到一个问题&#xff0c;在 ElSelect 组件中设置 filterable 属性后&#xff0c;监测不到复制粘贴的内容&#xff0c;也就意味着不能调用接口&#xff0c;下拉框内容为空。 简要代码如下&#xff1a; <ElSelectstyle"width: 256px"multiplev-model{siteIdL…...

加密与数据提取:保护隐私的新途径

加密与数据提取&#xff1a;保护隐私的新途径 在数字化时代&#xff0c;数据已成为驱动社会进步和经济发展的关键要素。然而&#xff0c;随着数据量的爆炸性增长&#xff0c;个人隐私保护成为了一个亟待解决的问题。如何在利用数据价值的同时&#xff0c;确保个人隐私不被侵犯…...

博客摘录「 宋宝华:Linux文件读写(BIO)波澜壮阔的一生」2024年11月1日

同时内核会给第2页标识一个PageReadahead标记&#xff0c;意思就是如果app接着读第2页&#xff0c;就可以预判app在做顺序读&#xff0c;这样我们在app读第2页的时候&#xff0c;内核可以进一步异步预读。 每个bio对应的硬盘里面一块连续的位置&#xff0c;每一块硬盘里面连续…...

使用华为云数字人可以做什么

在数字化和智能化快速发展的今天&#xff0c;企业面临着如何提升客户体验、优化运营效率的挑战。华为云数字人作为一种创新的智能交互解决方案&#xff0c;为企业提供了全新的可能性&#xff0c;助力企业在各个领域实现智能化升级。 提升客户服务体验 华为云数字人能够模拟真…...

leetcode刷题记录——(十六)349. 两个数组的交集

&#xff08;一&#xff09;问题描述 . - 力扣&#xff08;LeetCode&#xff09;. - 备战技术面试&#xff1f;力扣提供海量技术面试资源&#xff0c;帮助你高效提升编程技能,轻松拿下世界 IT 名企 Dream Offer。https://leetcode.cn/problems/intersection-of-two-arrays/ …...

vue3实现规则编辑器

组件用于创建和编辑复杂的条件规则&#xff0c;支持添加、删除条件和子条件&#xff0c;以及选择不同的条件类型。 可实现json数据和页面显示的转换。 代码实现 &#xff1a; index.vue: <template><div class"allany-container"><div class"co…...

【快速上手】pyspark 集群环境下的搭建(Standalone模式)

目录 前言 &#xff1a; 一、spark运行的五种模式 二、 安装步骤 安装前准备 1.第一步&#xff1a;安装python 2.第二步&#xff1a;在bigdata01上安装spark 3.第三步&#xff1a;同步bigdata01中的spark到bigdata02和03上 三、集群启动/关闭 四、打开监控界面验证 前…...

中文NLP地址要素解析【阿里云:天池比赛】

比赛地址&#xff1a;中文NLP地址要素解析 https://tianchi.aliyun.com/notebook/467867?spma2c22.12281976.0.0.654b265fTnW3lu长期赛&#xff1a; 分数:87.7271 排名&#xff1a;长期赛:56&#xff08;本次&#xff09;/6990&#xff08;团体或个人&#xff09;方案&#xf…...

使用AddressSanitizer内存检测

修改cmakelist.txt&#xff0c;在project(xxxx)后面追加&#xff1a; option(MEM_CHECK "memory check with AddressSanitizer" OFF) if(MEM_CHECK)set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitizeaddress")set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS…...

11月1日星期五今日早报简报微语报早读

11月1日星期五&#xff0c;农历十月初一&#xff0c;早报#微语早读。 1、六大行今日起实施存量房贷利率新机制。 2、谷歌被俄罗斯罚款35位数&#xff0c;罚款远超全球GDP。 3、山西吕梁&#xff1a;女性35岁前登记结婚&#xff0c;给予1500元奖励。 4、我国人均每日上网时间…...

实用篇:Postman历史版本下载

postman历史版本下载步骤 1.官方历史版本发布信息 2.点进去1中的链接,往下滑动;选择你想要的版本 例如下载v11.18版本 3.根据操作系统选择 mac:mac系统postman下载 window:window系统postman下载 4.在old version里找到对应版本下载即可 先点击download 再点击free downlo…...

微服务实战系列之玩转Docker(十七)

导览 前言Q&#xff1a;如何实现etcd数据的可视化管理一、创建etcd集群1. 节点定义2. 集群成员2.1 docker ps2.2 docker exec2.3 etcdctl member list 二、发布数据1. 添加数据2. 数据共享 三、可视化管理1. ETCD Keeper入门1.1 简介1.2 安装1.2.1 定义compose.yml1.2.2 启动ke…...

操作系统-实验报告单(1)

目录 1 实验目标 2 实验工具 3 实验内容、实验步骤及实验结果 一、安装虚拟机及Ubuntu 5、*存在虚拟机不能安装的问题 二、Ubuntu基本操作 1、桌面操作 2、终端命令行操作 三、在Ubuntu下运行C程序 3、*Ubuntu中编写一个Hello.c的主要程序 4 实验总结 实 验 报 告…...

rom定制系列------小米8青春版定制安卓14批量线刷固件 原生系统

&#x1f49d;&#x1f49d;&#x1f49d;小米8青春版。机型代码platina。官方最终版为 12.5.1安卓10的版本。客户需要安卓14的固件以便使用他们的软件。根据测试&#xff0c;原生pixeExpe固件适配兼容性较好。为方便客户批量进行刷写。修改固件为可fast批量刷写。整合底层分区…...

CATIA许可证常见问题解答

在使用CATIA软件的过程中&#xff0c;许可证问题常常是用户关心的焦点。为了帮助大家更好地理解和解决这些问题&#xff0c;我们整理了一份CATIA许可证常见问题解答&#xff0c;希望能为您提供便捷的参考。 问题一&#xff1a;如何激活CATIA许可证&#xff1f; 解答&#xff1a…...

PySpark Standalone 集群部署教程

目录 1. 环境准备 1.1 配置免密登录 2. 下载并配置Spark 3. 配置Spark集群 3.1 配置spark-env.sh 3.2 配置spark-defaults.conf 3.3 设置Master和Worker节点 3.4 设配置log4j.properties 3.5 同步到所有Worker节点 4. 启动Spark Standalone集群 4.1 启动Master节点 …...

【源码+文档】基于SpringBoot+Vue旅游网站系统【提供源码+答辩PPT+参考文档+项目部署】

作者简介&#xff1a;✌CSDN新星计划导师、Java领域优质创作者、掘金/华为云/阿里云/InfoQ等平台优质作者、专注于Java技术领域和学生毕业项目实战,高校老师/讲师/同行前辈交流。✌ 主要内容&#xff1a;&#x1f31f;Java项目、Python项目、前端项目、PHP、ASP.NET、人工智能…...

9.排队模型-M/M/1

1.排队模型 在Excel中建立排队模型可以帮助分析系统中的客户流动和服务效率。以下是如何构建简单排队模型的步骤&#xff1a; 1.确定模型参数 到达率&#xff08;λ&#xff09;&#xff1a;客户到达系统的平均速率&#xff08;例如每小时到达的客户数&#xff09;。服务率&…...

stm32G473的flash模式是单bank还是双bank?

今天突然有人stm32G473的flash模式是单bank还是双bank&#xff1f;由于时间太久&#xff0c;我真忘记了。搜搜发现&#xff0c;还真有人和我一样。见下面的链接&#xff1a;https://shequ.stmicroelectronics.cn/forum.php?modviewthread&tid644563 根据STM32G4系列参考手…...

Zustand 状态管理库:极简而强大的解决方案

Zustand 是一个轻量级、快速和可扩展的状态管理库&#xff0c;特别适合 React 应用。它以简洁的 API 和高效的性能解决了 Redux 等状态管理方案中的繁琐问题。 核心优势对比 基本使用指南 1. 创建 Store // store.js import create from zustandconst useStore create((set)…...

中南大学无人机智能体的全面评估!BEDI:用于评估无人机上具身智能体的综合性基准测试

作者&#xff1a;Mingning Guo, Mengwei Wu, Jiarun He, Shaoxian Li, Haifeng Li, Chao Tao单位&#xff1a;中南大学地球科学与信息物理学院论文标题&#xff1a;BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs论文链接&#xff1a;https://arxiv.…...

测试markdown--肇兴

day1&#xff1a; 1、去程&#xff1a;7:04 --11:32高铁 高铁右转上售票大厅2楼&#xff0c;穿过候车厅下一楼&#xff0c;上大巴车 &#xffe5;10/人 **2、到达&#xff1a;**12点多到达寨子&#xff0c;买门票&#xff0c;美团/抖音&#xff1a;&#xffe5;78人 3、中饭&a…...

反射获取方法和属性

Java反射获取方法 在Java中&#xff0c;反射&#xff08;Reflection&#xff09;是一种强大的机制&#xff0c;允许程序在运行时访问和操作类的内部属性和方法。通过反射&#xff0c;可以动态地创建对象、调用方法、改变属性值&#xff0c;这在很多Java框架中如Spring和Hiberna…...

Spring Boot面试题精选汇总

&#x1f91f;致敬读者 &#x1f7e9;感谢阅读&#x1f7e6;笑口常开&#x1f7ea;生日快乐⬛早点睡觉 &#x1f4d8;博主相关 &#x1f7e7;博主信息&#x1f7e8;博客首页&#x1f7eb;专栏推荐&#x1f7e5;活动信息 文章目录 Spring Boot面试题精选汇总⚙️ **一、核心概…...

如何理解 IP 数据报中的 TTL?

目录 前言理解 前言 面试灵魂一问&#xff1a;说说对 IP 数据报中 TTL 的理解&#xff1f;我们都知道&#xff0c;IP 数据报由首部和数据两部分组成&#xff0c;首部又分为两部分&#xff1a;固定部分和可变部分&#xff0c;共占 20 字节&#xff0c;而即将讨论的 TTL 就位于首…...

使用 SymPy 进行向量和矩阵的高级操作

在科学计算和工程领域&#xff0c;向量和矩阵操作是解决问题的核心技能之一。Python 的 SymPy 库提供了强大的符号计算功能&#xff0c;能够高效地处理向量和矩阵的各种操作。本文将深入探讨如何使用 SymPy 进行向量和矩阵的创建、合并以及维度拓展等操作&#xff0c;并通过具体…...

【电力电子】基于STM32F103C8T6单片机双极性SPWM逆变(硬件篇)

本项目是基于 STM32F103C8T6 微控制器的 SPWM(正弦脉宽调制)电源模块,能够生成可调频率和幅值的正弦波交流电源输出。该项目适用于逆变器、UPS电源、变频器等应用场景。 供电电源 输入电压采集 上图为本设计的电源电路,图中 D1 为二极管, 其目的是防止正负极电源反接, …...

Vue 模板语句的数据来源

&#x1f9e9; Vue 模板语句的数据来源&#xff1a;全方位解析 Vue 模板&#xff08;<template> 部分&#xff09;中的表达式、指令绑定&#xff08;如 v-bind, v-on&#xff09;和插值&#xff08;{{ }}&#xff09;都在一个特定的作用域内求值。这个作用域由当前 组件…...