Jast blog
Jast-zsh
2024-04-08
目录

Ambari自定义服务开发-自定义开发安装包整合到Ambari-Server中

[toc]

# RPM重新编译

# 安装rpmrebuild

下载地址：https://download.csdn.net/download/zhangshenghang/88964872 (opens new window)

# 创建目录
mkdir rpmrebuild
# 进入目录
cd rpmrebuild
# 将下载的tar.gz文件上传到rpmrebuild目录
tar -zxvf rpmrebuild-2.11.tar.gz
# 编译
make && make install
1
2
3
4
5
6
7
8

# 安装rpm-build

yum install -y rpm-build
1

# 自定义服务同步到服务器

将我们集成的服务DORIS,同步到下面这个目录

/var/lib/ambari-server/resources/stacks/HDP/3.1/services
1

同步完可以看到目录DORIS

$ ll
total 0
drwxr-xr-x. 3 root root   47 Jan 19 15:33 AMBARI_METRICS
drwxr-xr-x. 3 root root   47 Jan 19 15:33 ATLAS
drwxr-xr-x. 6  501 games 220 Mar 14 16:33 DORIS
drwxr-xr-x. 2 root root   26 Jan 19 15:33 HBASE
drwxr-xr-x. 3 root root  127 Jan 19 15:33 HDFS
drwxr-xr-x. 2 root root  106 Jan 19 15:33 HIVE
drwxr-xr-x. 2 root root   26 Jan 19 15:33 KAFKA
drwxr-xr-x. 2 root root   26 Jan 19 15:33 KNOX
drwxr-xr-x. 2 root root   26 Jan 19 15:33 PIG
drwxr-xr-x. 3 root root  127 Jan 19 15:33 RANGER
drwxr-xr-x. 4 root root   61 Jan 19 15:33 RANGER_KMS
drwxr-xr-x. 3 root root   47 Jan 19 15:33 SPARK2
drwxr-xr-x. 2 root root   26 Jan 19 15:33 TEZ
drwxr-xr-x. 2 root root   26 Jan 19 15:33 YARN
drwxr-xr-x. 2 root root   26 Jan 19 15:33 ZOOKEEPER
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# 生成需要添加的配置

这个命令是用于添加配置到.spec文件，.spec文件一会在编译时候会自动进入

find /var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS -type f -printf '%%attr(0644, root, root) "%p"\n' -o -type d -printf '%%dir %%attr(0755, root, root) "%p"\n' > doris.txt
1

查看内容

%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/metrics.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/configuration"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/configuration/fe.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/configuration/doris-env.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/configuration/be.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/alerts.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/role_command_order.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/quicklinks"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/quicklinks/quicklinks.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/metainfo.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/widgets.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/themes"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/themes/theme.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/alerts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/alerts/alert_fe_pid_exists.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/alerts/alert_be_pid_exists.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/params.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/changeBEOsConf.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/changeOsConf.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/status_params.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/frontend_follower.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/service_check.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/be.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/doris_client.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/frontend_observer.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/scripts/fe.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/templates"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/templates/fe.conf.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/templates/test-common.conf.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/DORIS/package/templates/be.conf.j2"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

在安装Ambari Server时我们会将安装包放到httpd服务中，查看我们的ambari-server版本,可以看到ambari-server-2.7.5.0-72.x86_64.rpm

$ ls /var/www/html/ambari/centos7/2.7.5.0-72/ambari/ambari-*
ambari-agent-2.7.5.0-72.x86_64.rpm
ambari-infra-manager-2.7.5.0-72.noarch.rpm
ambari-infra-solr-2.7.5.0-72.noarch.rpm
ambari-infra-solr-client-2.7.5.0-72.noarch.rpm
ambari-logsearch-logfeeder-2.7.5.0-72.noarch.rpm
ambari-logsearch-portal-2.7.5.0-72.noarch.rpm
ambari-metrics-collector-2.7.5.0-72.x86_64.rpm
ambari-metrics-common-2.7.5.0-72.noarch.rpm
ambari-metrics-grafana-2.7.5.0-72.x86_64.rpm
ambari-metrics-hadoop-sink-2.7.5.0-72.x86_64.rpm
ambari-metrics-monitor-2.7.5.0-72.x86_64.rpm
ambari-server-2.7.5.0-72.x86_64.rpm
1
2
3
4
5
6
7
8
9
10
11
12
13

重新rebuild编译ambari-server，ambari-server-2.7.5.0-72.x86_64就是上面我们看到的rpm包，去掉.rpm后缀

rpmrebuild -e ambari-server-2.7.5.0-72.x86_64
Do you want to continue ? (y/N) y
Do you want to change release number ? (y/N) n # 输入n，这时会进入编辑页面(即编辑 .spec 文件)，按i可以进入编辑,进入大概7100多行，添加上面我们生成的配置内容,:wq保存（和在vi编辑器里面编辑一样）
Do you want to continue ? (y/N) y
result: /root/rpmbuild/RPMS/x86_64/ambari-server-2.7.5.0-72.x86_64.rpm
1
2
3
4
5

编译完成之后会生成文件/root/rpmbuild/RPMS/x86_64/ambari-server-2.7.5.0-72.x86_64.rpm

查看编译后的是否包含DORIS

rpm -qpl  /root/rpmbuild/RPMS/x86_64/ambari-server-2.7.5.0-72.x86_64.rpm  |grep DORIS
1

# 验证RPM文件是否生效

  1. 停止Ambari Server，ambari-server stop
  2. 卸载 ambari-server 服务：yum remove ambari-server
  3. 安装 ambari-server 服务：yum install -y /root/rpmbuild/RPMS/x86_64/ambari-server-2.7.5.0-72.x86_64.rpm
  4. 重新设置 ambari-server 配置：ambari-server setup
 ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2 # 选择2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/jdk1.8.0_351 # 输入安装的jdk路径
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): 172.16.24.174
Port (3306): 3306
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata): 
Re-enter password: 
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
....ambari-admin-2.7.5.0.72.jar
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
  1. 启动服务：ambari-server start
  2. 登录Ambari Server Web UI，安装DORIS服务，测试功能是否正常。

# 如何通过yum install ambari-server直接安装?

上面我可以可以通过指定文件进行安装

yum install -y /root/rpmbuild/RPMS/x86_64/ambari-server-2.7.5.0-72.x86_64.rpm
1

如果想直接通过 yum install ambari-server 安装需要对本地配置的yum仓库下的repodata进行重新生成，该目录会校验安装包是否被修改。

我们查看ambari.repo可以看到ambari的库所在目录,可以看到下载目录ambari/centos7/2.7.5.0-72，绝对目录为：/var/www/html/ambari/centos7/2.7.5.0-72

# cat /etc/yum.repos.d/ambari.repo 
[ambari]
name=ambari
baseurl=http://bigdata-01/ambari/centos7/2.7.5.0-72
enabled=1
gpgcheck=0
1
2
3
4
5
6

目录结构为

[root@bigdata-01 2.7.5.0-72]# pwd
/var/www/html/ambari/centos7/2.7.5.0-72
[root@bigdata-01 2.7.5.0-72]# ll
total 52
drwxr-xr-x 2 1001 users 4096 Mar 20 10:49 ambari
-rw-r--r-- 1 1001 users  375 Mar 21 18:45 ambari.repo
-rw-r--r-- 1 1001 users 1608 Dec 18  2019 artifacts.txt
-rw-r--r-- 1 1001 users  585 Dec 18  2019 build.id
-rw-r--r-- 1 1001 users 1667 Dec 18  2019 build_metadata.txt
-rw-r--r-- 1 1001 users  500 Dec 18  2019 hotfix_index.html
-rw-r--r-- 1 1001 users  505 Dec 18  2019 index.html
-rw-r--r-- 1 1001 users  497 Dec 18  2019 private_index.html
-rw-r--r-- 1 1001 users  496 Dec 18  2019 public_index.html
drwxr-xr-x 2 root root  4096 Mar 21 18:57 repodata
drwxr-xr-x 2 1001 users 4096 Dec 18  2019 RPM-GPG-KEY
drwxr-xr-x 2 1001 users 4096 Dec 18  2019 smartsense
drwxr-xr-x 5 1001 users 4096 Dec 18  2019 tars
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

进入 ambari 目录,可以看到ambari-server-2.7.5.0-72.x86_64.rpm，将我们修改后的文件替换掉之前的这个

# cd ambari
[root@bigdata-01 ambari]# ls
ambari-agent-2.7.5.0-72.x86_64.rpm                ambari-metrics-collector-2.7.5.0-72.x86_64.rpm
ambari-infra-manager-2.7.5.0-72.noarch.rpm        ambari-metrics-common-2.7.5.0-72.noarch.rpm
ambari-infra-solr-2.7.5.0-72.noarch.rpm           ambari-metrics-grafana-2.7.5.0-72.x86_64.rpm
ambari-infra-solr-client-2.7.5.0-72.noarch.rpm    ambari-metrics-hadoop-sink-2.7.5.0-72.x86_64.rpm
ambari-logsearch-logfeeder-2.7.5.0-72.noarch.rpm  ambari-metrics-monitor-2.7.5.0-72.x86_64.rpm
ambari-logsearch-portal-2.7.5.0-72.noarch.rpm     ambari-server-2.7.5.0-72.x86_64.rpm
1
2
3
4
5
6
7
8

替换之后，此时我们通过yum install ambari-server安装会提示安装失败，这里就是校验出现了问题，下面我们讲处理方法

image-20240322100240618

处理方法

  1. 删除/var/www/html/ambari/centos7/2.7.5.0-72/repodata目录
  2. 安装createrepo，sudo yum install -y createrepo
  3. 在/var/www/html/ambari/centos7/2.7.5.0-72/目录下运行createrepo .，此时如果正常会生成repodata目录
  4. 刷新yum缓存，让我们刚刚生成的repodata生效
sudo yum clean all
sudo yum makecache
1
2
  1. 安装服务yum install -y ambari-server
  2. 成功安装
上次更新: 2024/04/08, 10:55:49
Ambari自定义服务开发-美化(增强型)配置参数设置
Ambari自定义服务开发-自定义脚本运行

← Ambari自定义服务开发-自定义脚本运行→

最近更新
01
users xml配置文件详解
07-12
02
Protobuf详细介绍
07-12
03
Java Maven中自动代码检查插件
07-12
更多文章>
Theme by Vdoing | Copyright © 2016-2024 Jast-zsh | MIT License