重庆分公司,新征程启航
为企业提供网站建设、域名注册、服务器等服务
1.下载软件:
我们提供的服务有:网站设计、网站制作、微信公众号开发、网站优化、网站认证、庐山ssl等。为上千家企事业单位解决了网站和推广的问题。提供周到的售前咨询和贴心的售后服务,是有科学管理、有技术的庐山网站制作公司
wget http://apache.fayea.com/pig/pig-0.15.0/pig-0.15.0.tar.gz
2.解压
tar -zxvf pig-0.15.0.tar.gz
mv pig-0.15.0 /usr/local/
ln -s pig-0.15.0 pig
3.配置环境变量:
export PATH=PATH=$HOME/bin:/usr/local/hadoop/bin:/usr/local/hadoop/sbin:/usr/local/pig/bin:$PATH;
export PIG_CLASSPATH=/usr/local/hadoop/etc/hadoop;
4.进入grunt shell:
以本地模式登录pig: 该方式的所有文件和执行过程都在本地,一般用于测试程序
[hadoop@host61 ~]$ pig -x local
15/10/03 01:14:09 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/10/03 01:14:09 INFO pig.ExecTypeProvider: Picked LOCAL as the ExecType
2015-10-03 01:14:09,756 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0 (r1682971) compiled Jun 01 2015, 11:44:35
2015-10-03 01:14:09,758 [main] INFO org.apache.pig.Main - Logging error messages to: /home/hadoop/pig_1443860049744.log
2015-10-03 01:14:10,133 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
2015-10-03 01:14:12,648 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2015-10-03 01:14:12,656 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-10-03 01:14:12,685 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2015-10-03 01:14:13,573 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
grunt>
以Mapreduce模式登录:实际工作模式:
[hadoop@host63 ~]$ pig
15/10/03 02:11:54 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/10/03 02:11:54 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
15/10/03 02:11:54 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2015-10-03 02:11:55,086 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0 (r1682971) compiled Jun 01 2015, 11:44:35
2015-10-03 02:11:55,087 [main] INFO org.apache.pig.Main - Logging error messages to: /home/hadoop/pig_1443863515062.log
2015-10-03 02:11:55,271 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
2015-10-03 02:11:59,735 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-10-03 02:11:59,740 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2015-10-03 02:11:59,742 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://host61:9000/
2015-10-03 02:12:06,256 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-10-03 02:12:06,257 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: host61:9001
2015-10-03 02:12:06,265 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
grunt>
5.pig的运行方式有如下三种:
1.脚本
2.grunt
3.嵌入式
6.登录pig,并使用常用的命令:
[hadoop@host63 ~]$ pig
15/10/03 06:01:01 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/10/03 06:01:01 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
15/10/03 06:01:01 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2015-10-03 06:01:01,412 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0 (r1682971) compiled Jun 01 2015, 11:44:35
2015-10-03 06:01:01,413 [main] INFO org.apache.pig.Main - Logging error messages to: /home/hadoop/pig_1443877261408.log
2015-10-03 06:01:01,502 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
2015-10-03 06:01:03,657 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-10-03 06:01:03,657 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2015-10-03 06:01:03,662 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://host61:9000/
2015-10-03 06:01:05,968 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-10-03 06:01:05,968 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: host61:9001
2015-10-03 06:01:05,979 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
grunt> help
Commands:
File system commands:
fs
Diagnostic commands:
describe explain [-script [-param_file -script - Explain the entire script. -out - Store the output into directory rather than print to stdout. -brief - Don't expand nested plans (presenting a smaller graph for overview). -dot - Generate the output in .dot format. Default is text format. -xml - Generate the output in .xml format. Default is text format. -param -param_file alias - Alias to explain. dump Utility Commands: exec [-param