組件配置

您可以使用以下任意一種方式，配置歸一化組件參數(shù)。

方式一：可視化方式

方式二：PAI命令方式

示例

• 數(shù)據(jù)生成

  drop table if exists normalize_test_input;
  create table normalize_test_input(
      col_string string,
      col_bigint bigint,
      col_double double,
      col_boolean boolean,
      col_datetime datetime);
  insert overwrite table normalize_test_input
  select
      *
  from
  (
      select
          '01' as col_string,
          10 as col_bigint,
          10.1 as col_double,
          True as col_boolean,
          cast('2016-07-01 10:00:00' as datetime) as col_datetime
      union all
          select
              cast(null as string) as col_string,
              11 as col_bigint,
              10.2 as col_double,
              False as col_boolean,
              cast('2016-07-02 10:00:00' as datetime) as col_datetime
      union all
          select
              '02' as col_string,
              cast(null as bigint) as col_bigint,
              10.3 as col_double,
              True as col_boolean,
              cast('2016-07-03 10:00:00' as datetime) as col_datetime
      union all
          select
              '03' as col_string,
              12 as col_bigint,
              cast(null as double) as col_double,
              False as col_boolean,
              cast('2016-07-04 10:00:00' as datetime) as col_datetime
      union all
          select
              '04' as col_string,
              13 as col_bigint,
              10.4 as col_double,
              cast(null as boolean) as col_boolean,
              cast('2016-07-05 10:00:00' as datetime) as col_datetime
      union all
          select
              '05' as col_string,
              14 as col_bigint,
              10.5 as col_double,
              True as col_boolean,
              cast(null as datetime) as col_datetime
  ) tmp;

• PAI命令行

  drop table if exists normalize_test_input_output;
  drop table if exists normalize_test_input_model_output;
  PAI -name Normalize
      -project algo_public
      -DoutputParaTableName="normalize_test_input_model_output"
      -Dlifecycle="28"
      -DoutputTableName="normalize_test_input_output"
      -DinputTableName="normalize_test_input"
      -DselectedColNames="col_double,col_bigint"
      -DkeepOriginal="true";
  drop table if exists normalize_test_input_output_using_model;
  drop table if exists normalize_test_input_output_using_model_model_output;
  PAI -name Normalize
      -project algo_public
      -DoutputParaTableName="normalize_test_input_output_using_model_model_output"
      -DinputParaTableName="normalize_test_input_model_output"
      -Dlifecycle="28"
      -DoutputTableName="normalize_test_input_output_using_model"
      -DinputTableName="normalize_test_input";

• 輸入說明

  normalize_test_input

  col_string	col_bigint	col_double	col_boolean	col_datetime
  01	10	10.1	true	2016-07-01 10:00:00
  NULL	11	10.2	false	2016-07-02 10:00:00
  02	NULL	10.3	true	2016-07-03 10:00:00
  03	12	NULL	false	2016-07-04 10:00:00
  04	13	10.4	NULL	2016-07-05 10:00:00
  05	14	10.5	true	NULL

• 輸出說明

  • normalize_test_input_output

    col_string	col_bigint	col_double	col_boolean	col_datetime	normalized_col_bigint	normalized_col_double
    01	10	10.1	true	2016-07-01 10:00:00	0.0	0.0
    NULL	11	10.2	false	2016-07-02 10:00:00	0.25	0.2499999999999989
    02	NULL	10.3	true	2016-07-03 10:00:00	NULL	0.5000000000000022
    03	12	NULL	false	2016-07-04 10:00:00	0.5	NULL
    04	13	10.4	NULL	2016-07-05 10:00:00	0.75	0.7500000000000011
    05	14	10.5	true	NULL	1.0	1.0

  • normalize_test_input_model_output

    feature	json
    col_bigint	{“name”: “normalize”, “type”:”bigint”, “paras”:{“min”:10, “max”: 14}}
    col_double	{“name”: “normalize”, “type”:”double”, “paras”:{“min”:10.1, “max”: 10.5}}

  • normalize_test_input_output_using_model

    col_string	col_bigint	col_double	col_boolean	col_datetime
    01	0.0	0.0	true	2016-07-01 10:00:00
    NULL	0.25	0.2499999999999989	false	2016-07-02 10:00:00
    02	NULL	0.5000000000000022	true	2016-07-03 10:00:00
    03	0.5	NULL	false	2016-07-04 10:00:00
    04	0.75	0.7500000000000011	NULL	2016-07-05 10:00:00
    05	1.0	1.0	true	NULL

  • normalize_test_input_output_using_model_model_output

    feature	json
    col_bigint	{“name”: “normalize”, “type”:”bigint”, “paras”:{“min”:10, “max”: 14}}
    col_double	{“name”: “normalize”, “type”:”double”, “paras”:{“min”:10.1, “max”: 10.5}}