问题导读： 01 Flink SQL 提升 源表自定义并行度 -- set parallelism within the ddl
CREATE TABLE Orders (
order_number BIGINT,
price DECIMAL(32,2),
buyer ROW<first_name STRING, last_name STRING>,
order_time TIMESTAMP(3)
) WITH (
'connector' = 'datagen',
'scan.parallelism' = '4'
);
-- or set parallelism via dynamic table option
SELECT * FROM Orders /*+ OPTIONS('scan.parallelism'='4') */; 复制代码 更多信息 https://nightlies.apache.org/fli ... /#scan-table-source FLIP-367: Support Setting Parallelism for Table/SQL Sources https://cwiki.apache.org/conflue ... on?pageId=263429150 可配置的 SQL Gateway Java 选项 更多信息 https://issues.apache.org/jira/browse/FLINK-33203 使用 SQL 提示配置不同的状态 TTL https://nightlies.apache.org/fli ... ts/#state-ttl-hints ），以更灵活的方式直接在查询中为常规连接和分组聚合指定自定义 TTL 值。-- set state ttl for join
SELECT /*+ STATE_TTL('Orders'= '1d', 'Customers' = '20d') */ *
FROM Orders LEFT OUTER JOIN Customers
ON Orders.o_custkey = Customers.c_custkey;
-- set state ttl for aggregation
SELECT /*+ STATE_TTL('o' = '1d') */ o_orderkey, SUM(o_totalprice) AS revenue
FROM Orders AS o
GROUP BY o_orderkey;
复制代码 更多信息 https://nightlies.apache.org/fli ... ts/#state-ttl-hints FLIP-373: Support Configuring Different State TTLs using SQL Hint https://cwiki.apache.org/conflue ... TTLs+using+SQL+Hint 函数和存储过程支持命名参数 public static class NamedArgumentsTableFunction extends TableFunction<Object> {
@FunctionHint(
output = @DataTypeHint("STRING"),
arguments = {
@ArgumentHint(name = "in1", isOptional = false, type = @DataTypeHint("STRING")),
@ArgumentHint(name = "in2", isOptional = true, type = @DataTypeHint("STRING")),
@ArgumentHint(name = "in3", isOptional = true, type = @DataTypeHint("STRING"))})
public void eval(String arg1, String arg2, String arg3) {
collect(arg1 + ", " + arg2 + "," + arg3);
}
}
复制代码 SELECT * FROM TABLE(myFunction(in1 => 'v1', in3 => 'v3', in2 => 'v2')) 复制代码 SELECT * FROM TABLE(myFunction(in1 => 'v1')) 复制代码 更多信息 https://nightlies.apache.org/fli ... s/#named-parameters FLIP-387: Support named parameters for functions and call procedures https://cwiki.apache.org/conflue ... and+call+procedures Window TVF 聚合功能 支持流模式下的 SESSION Window TVF -- session window with partition keys
SELECT * FROM TABLE(
SESSION(TABLE Bid PARTITION BY item, DESCRIPTOR(bidtime), INTERVAL '5' MINUTES));
-- apply aggregation on the session windowed table with partition keys
SELECT window_start, window_end, item, SUM(price) AS total_price
FROM TABLE(
SESSION(TABLE Bid PARTITION BY item, DESCRIPTOR(bidtime), INTERVAL '5' MINUTES))
GROUP BY item, window_start, window_end;
复制代码 更多信息 https://nightlies.apache.org/fli ... window-tvf/#session 新的 UDF 类型：AsyncScalarFunction 更多信息 FLIP-400: AsyncScalarFunction for asynchronous scalar function support https://cwiki.apache.org/conflue ... ar+function+support Regular Join 支持 MiniBatch 优化
更多信息 minibatch-regular-joins 文档. https://nightlies.apache.org/fli ... batch-regular-joins FLIP-415: Introduce a new join operator to support minibatch https://cwiki.apache.org/conflue ... o+support+minibatch 02 Runtime & Coordination 提升 更多信息 https://nightlies.apache.org/fli ... support-for-sources FLIP-379: Support dynamic source parallelism inference for batch jobs https://cwiki.apache.org/conflue ... ence+for+batch+jobs Flink Configuration 支持标准 YAML 格式 更多信息 flink-configuration-file 文档 https://nightlies.apache.org/fli ... -configuration-file FLIP-366: Support standard YAML for Flink configuration https://cwiki.apache.org/conflue ... textnavpagetreemode 在 Flink Web 上 Profiling JobManager/TaskManager
更多信息 https://nightlies.apache.org/fli ... debugging/profiler/ FLIP-375: Built-in cross-platform powerful java profiler https://cwiki.apache.org/confluence/x/64lEE 新增管理员 JVM 选项配置选项 更多信息 https://nightlies.apache.org/fli ... and-logging-options FLIP-397: Add config options for administrator JVM options https://cwiki.apache.org/conflue ... VM+options?src=jira 03 Checkpoints 提升 更多信息 https://issues.apache.org/jira/browse/FLINK-32514 [FLIP-309: Support using larger checkpointing interval when source is processing backlog] https://cwiki.apache.org/conflue ... +processing+backlog CheckpointsCleaner 并行清理单个检查点状态 更多信息 https://issues.apache.org/jira/browse/FLINK-33090 通过命令行客户端触发 Checkpoints ./bin/flink checkpoint $JOB_ID [-full] 复制代码 更多信息 https://issues.apache.org/jira/browse/FLINK-6755 04 Connector API提升 与 Source API 一致的 SinkV2 新接口 更多信息 https://issues.apache.org/jira/browse/FLINK-33973 FLIP-372: Enhance and synchronize Sink API to match the Source API https://cwiki.apache.org/conflue ... atch+the+Source+API 用于跟踪Committables状态的新Committer指标 更多信息 https://issues.apache.org/jira/browse/FLINK-25857 FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink https://cwiki.apache.org/conflue ... PhaseCommittingSink 05 重要API弃用 Flink's org.apache.flink.api.common.time.Time[1] 现已被正式弃用，并将在 Flink 2.0 中删除。引入了支持 Duration 类的方法，以取代已废弃的基于 Time 的方法。 org.apache.flink.runtime.jobgraph.RestoreMode#LEGACY[2] 已被弃用。请使用 RestoreMode#CLAIM[3] 或 RestoreMode#NO_CLAIM[4] 模式，以在还原时获得清晰的状态文件所有权。 旧的解决模式兼容性的方法已被弃用，请参考迁移说明迁移至新方法：Migrating from deprecated TypeSerializerSnapshot#resolveSchemaCompatibility(TypeSerializer newSerializer) before Flink 1.19.[5] 通过硬代码配置序列化行为已被弃用，例如 ExecutionConfig#enableForceKryo[6] 。请使用选 pipeline.serialization-config 、pipeline.force-avr 、pipeline.force-kryo 和 pipeline.generic-types。实例级序列化器的注册已被弃用，请使用类级序列化器。 除了getString(String key, String defaultValue)[7] 和 setString(String key, String value)[8] ，我们已废弃所有 setXxx 和 getXxx 方法，如：setInteger 、setLong 、getInteger 和 getLong 等。建议用户和开发人员使用以 ConfigOption 代替字符串作为键的 get 和 set 方法。 StreamExecutionEnvironment 、CheckpointConfig 和 ExecutionConfig 中的非 ConfigOption 对象及其相应的 getter/setter 接口现已废弃。这些对象和方法计划在 Flink 2.0 中删除。已废弃的接口包括重启策略(RestartStrategy)、检查点存储(CheckpointStorage)和状态后端(StateBackend)的getter 和 setter 方法。 org.apache.flink.api.common.functions.RuntimeContext#getExecutionConfig [9] 现已被正式弃用，并将在 Flink 2.0 中删除。请使用 getGlobalJobParameters()[10] 或 isObjectReuseEnabled()[11] 。 org.apache.flink.api.common.functions.RichFunction#open(Configuration parameters)[12] 方法已被弃用，并将在未来版本中删除。我们鼓励用户迁移到新的 RichFunction#open(OpenContext openContext)[13] 。 org.apache.flink.configuration.AkkaOptions[14] 已被弃用，取而代之的是 RpcOptions[15] 。 网页链接： https://github.com/apache/flink/ ... mmon/time/Time.java https://github.com/apache/flink/ ... estoreMode.java#L40 https://github.com/apache/flink/ ... estoreMode.java#L31 https://github.com/apache/flink/ ... estoreMode.java#L34 https://nightlies.apache.org/fli ... schemacompatibility https://github.com/apache/flink/ ... ionConfig.java#L643 https://github.com/apache/flink/ ... iguration.java#L176 https://github.com/apache/flink/ ... iguration.java#L220 https://github.com/apache/flink/ ... meContext.java#L191 https://github.com/apache/flink/ ... meContext.java#L208 https://github.com/apache/flink/ ... meContext.java#L216 https://github.com/apache/flink/ ... chFunction.java#L76 https://github.com/apache/flink/ ... hFunction.java#L118 https://github.com/apache/flink/ ... on/AkkaOptions.java https://github.com/apache/flink/ ... ion/RpcOptions.java 06 升级说明 https://nightlies.apache.org/fli ... e-notes/flink-1.19/ ）获取更多的升级时需要的改动与可能的问题列表细节。https://mp.weixin.qq.com/s/DxoI38o7TpzpMXto8ep6ow 最新经典文章，欢迎关注公众号
/2 
京ICP备2020039040号
简书网举报电话：021-34700000
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
千斤顶
显身卡
