<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://zhihaol.eu.org/feed.xml" rel="self" type="application/atom+xml"/><link href="https://zhihaol.eu.org/" rel="alternate" type="text/html" hreflang="en"/><updated>2026-06-03T11:22:56+00:00</updated><id>https://zhihaol.eu.org/feed.xml</id><title type="html">Zhihao LIU</title><subtitle>Climate &amp; energy data scientist building reproducible ML workflows and decision tools from climate and geospatial data. </subtitle><entry><title type="html">The Agentic Brain: A Blueprint for Personal AI OS with LifeOS 5.0</title><link href="https://zhihaol.eu.org/blog/2026/lifeos-5-agentic-brain/" rel="alternate" type="text/html" title="The Agentic Brain: A Blueprint for Personal AI OS with LifeOS 5.0"/><published>2026-03-08T00:00:00+00:00</published><updated>2026-03-08T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2026/lifeos-5-agentic-brain</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2026/lifeos-5-agentic-brain/"><![CDATA[<p><img src="/assets/img/lifeos-5-logo.svg" alt="LifeOS 5.0 Logo" class="img-fluid rounded z-depth-1"/></p> <p>The dream of a “Second Brain” has long been synonymous with digital hoarding—a static warehouse of notes, PDFs, and half-remembered ideas. But in the era of Large Language Models (LLMs), <strong>the archive is dead</strong>. A brain that only stores is not a brain; it is a graveyard. For a personal operating system to be truly “alive,” it must evolve from passive storage to <strong>active agency</strong>.</p> <p>LifeOS 5.0 marks this transition. It is no longer a tool for Personal Knowledge Management (PKM); it is a <strong>Blueprint for a Personal AI Operating System</strong>. By decoupling logic (the Kernel) from storage (the Obsidian Vault), we have built a system that doesn’t just remember—it perceives, associates, and executes. <strong>Memory is not merely stored data, but the continuity of agency.</strong></p> <h3 id="mimicking-the-human-loop-the-cognitive-architecture">Mimicking the Human Loop: The Cognitive Architecture</h3> <p>The architectural novelty of LifeOS 5.0 does not lie in adding yet another retrieval layer. It lies in <strong>redefining what memory is</strong>. Conventional agent memory systems treat memory as an external resource that the agent retrieves in order to maintain response continuity. LifeOS-AMEM proposes a different model: memory as a <strong>co-evolving shared brain</strong> jointly inhabited by the human and the agent.</p> <style>
  .mermaid {
    max-width: 50%;
    margin-left: auto;
    margin-right: auto;
  }

  .mermaid svg {
    width: 100% !important;
    height: auto !important;
  }
</style> <pre><code class="language-mermaid">flowchart LR
    subgraph O["Agent-Centric Memory"]
        O1[Human] --&gt; O2[Agent]
        O2 --&gt; O3[Retrieve Memory]
        O3 --&gt; O2
        O2 --&gt; O4[Response]
    end

    subgraph L["LifeOS-AMEM"]
        L1[Human] &lt;--&gt; L2[Shared Brain&lt;br/&gt;living markdown memory]
        L3[Agent] &lt;--&gt; L2
        L2 --&gt; L4[sense / recall]
        L4 --&gt; L3
        L3 --&gt; L5[commit / maintain]
        L5 --&gt; L2
    end
</code></pre> <p><em>Figure X. LifeOS-AMEM transforms memory from an agent-centric retrieval target into a co-evolving human-agent shared brain.</em></p> <p><em>Conventional agent memory systems treat memory as an external resource retrieved by the agent to improve response continuity. In contrast, LifeOS-AMEM models memory as a shared cognitive substrate jointly inhabited by the human and the agent. Through <code class="language-plaintext highlighter-rouge">sense</code> and <code class="language-plaintext highlighter-rouge">recall</code>, the agent reads the current and historical state of this shared brain; through <code class="language-plaintext highlighter-rouge">commit</code> and <code class="language-plaintext highlighter-rouge">maintain</code>, it writes back governed updates, allowing memory to co-evolve as a digital twin rather than remain a passive retrieval store.</em></p> <h3 id="mimicking-the-human-neural-ulid-as-semantic-identity">Mimicking the Human Neural: ULID as Semantic Identity</h3> <p>How does a stateless AI agent maintain a coherent “grip” on a sprawling filesystem? In LifeOS 5.0, the <strong>ULID (Universally Unique Lexicographically Sortable Identifier)</strong> acts as a <strong>Neural Anchor</strong>.</p> <p>By embedding these identifiers into the fabric of the Markdown body, we create a permanent link between physical text and the database’s structured memory. This identity allows the agent to track a thought’s evolution across years and contexts without losing its essence. In this architecture, <strong>identity is the precursor to deep context</strong>, enabling the system to treat every block of text as a living, evolving neuron.</p> <h3 id="mimicking-the-human-sleep-consolidation--contextual-pruning">Mimicking the Human Sleep: Consolidation &amp; Contextual Pruning</h3> <p>Feeding an entire knowledge base into a prompt is the equivalent of trying to think while someone reads a dictionary at you. LifeOS 5.0 solves this by mimicking the <strong>Synaptic Pruning</strong> that occurs during human sleep.</p> <p>Utilizing the Obsidian CLI, the agent performs surgical extractions around specific ULID anchors. During “Governance Distillation” sessions—the system’s equivalent of REM sleep—the agent consolidates fragmented notes and prunes away “Context Noise.” By pulling only the relevant semantic neighborhood, we achieve <strong>Cognitive Load Balancing</strong>, ensuring the system hears only the signal, never the static.</p> <h3 id="conclusion-the-agentic-brain-as-a-functional-neuron">Conclusion: The Agentic Brain as a Functional Neuron</h3> <p>LifeOS 5.0 represents the end of the “Note-taking” era. We have moved beyond the storage of files into the era of <strong>Agentic Personal Operating Systems</strong>.</p> <p>The <strong>Agentic Brain</strong> is no longer a tool you use; it is a functional neuron within your own cognitive ecosystem. It responds, associates, and evolves in tandem with your intent. Your vault is no longer a static topology of files; it is an active, breathing extension of your own mind—a teammate in the ultimate pursuit of digital agency.</p>]]></content><author><name></name></author><category term="AI"/><category term="LifeOS"/><category term="Architecture"/><summary type="html"><![CDATA[LifeOS 5.0 transitions from passive digital hoarding to an active agentic brain, utilizing ULID anchors and cognitive pruning to mimic human memory consolidation.]]></summary></entry><entry><title type="html">让 AI 看见你的大脑：Obsidian CLI 的价值与意义</title><link href="https://zhihaol.eu.org/blog/2026/obsidian-graph-rag/" rel="alternate" type="text/html" title="让 AI 看见你的大脑：Obsidian CLI 的价值与意义"/><published>2026-03-05T00:00:00+00:00</published><updated>2026-03-05T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2026/obsidian-graph-rag</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2026/obsidian-graph-rag/"><![CDATA[<p>当我们谈论“第二大脑”时，大多数人的直觉依然停留在“容器”的概念上：一个存放笔记、文档和碎片的数字仓库。然而，Obsidian 的本质从未仅仅是文件夹的堆砌，它的生命力源于点（Node）与线（Edge）的交织。</p> <h3 id="从文件夹到拓扑图">从文件夹到拓扑图</h3> <p>在传统的笔记软件里，文件夹是绝对的权威。但在 Obsidian 的语境下，文件夹只是为了照顾人类视觉习惯而存在的低维分类，真正的灵魂蕴含在 <code class="language-plaintext highlighter-rouge">[[links]]</code> 之中。这不再是静态的层级，而是一种“拓扑结构”——就像大脑皮层上的神经元，一个知识点之所以有意义，不在于它被放在哪个抽屉里，而在于它与多少个其他节点产生了链接。</p> <p>这本质上是一个知识图谱（Knowledge Graph）。当我们人类在 Obsidian 中漫游时，我们是在利用视觉去捕捉这些链接；而当 AI 介入时，我们需要一种更纯粹的方式，让它“看见”这种结构。</p> <h3 id="消除交互噪音为什么-ai-需要-cli">消除“交互噪音”：为什么 AI 需要 CLI？</h3> <p>图形界面（GUI）是为人类设计的。图标、按钮、窗口，这些都是为了适配人类眼球和手指的“噪音”。对于 AI Agent 而言，它不需要华丽的界面，它需要的是一种穿透表象、直接触达数据骨架的能力。</p> <p>这就是 Obsidian CLI 或 API 的核心价值。</p> <p>普通的 AI 只能通过读取文件系统来“阅读”笔记，那就像是在一个漆黑的图书馆里随机翻开几本书。而具备 CLI 能力的 AI，拥有的是整个图书馆的“实时上下文（Live Context）”。它不只是读到了文件里的文字，它读到了整个保险库（Vault）的脉搏。通过 CLI，AI 可以直接查询图谱的连通性：哪些笔记是孤岛？哪些节点是交通枢纽？</p> <p>这种能力的转变，让 AI 从一个“外部读者”变成了“内部观察者”。它看见的不再是死板的 <code class="language-plaintext highlighter-rouge">.md</code> 文件，而是你大脑思维的动态拓扑。</p> <h3 id="进化从第二大脑到联感大脑">进化：从第二大脑到“联感大脑”</h3> <p>我们记录笔记，不应该是为了埋葬知识。过去，笔记是死的数据；现在，在 AI 的驱动下，笔记应该是活的知识。</p> <p>当 AI 能够理解你笔记之间的拓扑关系，一种奇妙的“联感（Synesthesia）”便会产生。你输入一段新的感悟，AI 不是简单地搜索关键词，而是根据现有的神经链接，自动为你勾勒出潜在的关联。这种自动化的联想，不是靠生硬的算法推荐，而是顺着你已有的思维纹路生长出来的。</p> <p>这种从“第二大脑（静态存储）”向“联感大脑（动态生成）”的跨越，正是数字生命与个人知识管理融合的终极形态。</p> <h3 id="拓扑查询发现的新范式">拓扑查询：发现的新范式</h3> <p>最近的实践告诉我们，改变发现范式的关键在于“如何提问”。</p> <p>当我们使用像 <code class="language-plaintext highlighter-rouge">obsidian-graph-query</code> 这样的工具去查询“关系”而非“内容”时，我们实际上是在审视自己思维的盲区。AI 不再只是回答“你记过什么”，它开始回答“这两个看似无关的想法之间，是否存在一条隐藏的路径”。</p> <p>让 AI 看见你的大脑，不是为了让它替你思考，而是为了让它通过那面 CLI 的镜子，帮你照见那些你原本无法察觉的、关于智慧的万千链接。</p>]]></content><author><name></name></author><category term="AI"/><category term="Obsidian"/><category term="LifeOS"/><summary type="html"><![CDATA[当我们谈论“第二大脑”时，大多数人的直觉依然停留在“容器”的概念上：一个存放笔记、文档和碎片的数字仓库。然而，Obsidian 的本质从未仅仅是文件夹的堆砌，它的生命力源于点（Node）与线（Edge）的交织。]]></summary></entry><entry xml:lang="eng"><title type="html">Latest Publication: Snow Depth Downscaling in CRST</title><link href="https://zhihaol.eu.org/blog/2025/crst-publication/" rel="alternate" type="text/html" title="Latest Publication: Snow Depth Downscaling in CRST"/><published>2025-11-01T00:00:00+00:00</published><updated>2025-11-01T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2025/crst-publication</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2025/crst-publication/"><![CDATA[]]></content><author><name>Zhihao</name></author><category term="climate"/><category term="remote_sensing"/><category term="links"/><category term="links"/><category term="research"/><summary type="html"><![CDATA[Retrieving snow depth distribution by downscaling ERA5 Reanalysis with ICESat-2 laser altimetry.]]></summary></entry><entry xml:lang="eng"><title type="html">此心光明，亦复何言</title><link href="https://zhihaol.eu.org/blog/2024/yangming/" rel="alternate" type="text/html" title="此心光明，亦复何言"/><published>2024-01-28T00:00:00+00:00</published><updated>2024-01-28T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2024/yangming</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2024/yangming/"><![CDATA[<h2 id="格物致知">格物致知</h2> <p>“众物必有表里精粗，一草一木，皆涵至理，格物穷理”（朱熹）。理学认为，理，是天道运行的最高范畴，蕴藏在万事万物之中，“格天下之物，而后有天下之知”。在如此理学认识论基础上，王阳明16岁格竹七日，却一无所获。他认为朱熹解释不清楚理在何处。</p> <p>王阳明17岁时曾向娄谅求教，“圣人可学而致之”，每个人都可以祛遮成圣（目标）。而陆九渊认为“吾心即是宇宙（心即理）”，很好填补了程朱理学给王阳明带来的困惑。我即真理，追寻心的真相就可参透天理（方法）。如此，构成了王阳明心学的起点，“圣人之学，心学也“。</p> <ul> <li>格物穷理也是一种科学精神。理可以外求，但只做观察，缺少实验也无法至知。这是理学与西方实验主义科学的区别。</li> <li>佛教在唐朝过于繁荣，唐儒家韩愈站在道统问题的立场上反对佛教。面对佛教冲击，北宋思想家则开始接纳并深入研究佛教，并对先秦儒家文本进行重新解释，诞生了理学。此时，理学已经对佛道有诸多借鉴，比如静坐。而阳明心学则是对理学运动的再一次推动（是继承关系，而不是反对关系）。</li> </ul> <h2 id="心即理心外无物">心即理，心外无物</h2> <p>王阳明少年做诗，“山近月远觉月小，便道此山大于月，若有人眼大如天，还见山小月更阔。”王阳明数次科举不中，”世以不得第为耻，吾以不得第动心为耻“。王阳明有非常不一样的思考角度。</p> <p>34岁，王阳明在人生经历诸多挫折后，龙场悟道。王阳明把心即理，拓展解释为“心外无物、心外无事、心外无理”。也就是说，心与物同体，物不能离开心而存在，心也不能离开物存在。不必在事事物物上求理，在心外求理，而应该追求本心的开悟。</p> <ul> <li>《传习录》有：<strong>心之发为意，意之所在为物</strong>。也就是说，物存在人的心内，被意识感知到的存在才是物，它更接近于“心理存在”，接近于意义。物不单指客观物体，而是任何投影于内心的价值存在。</li> <li>《传习录》有一则《游南镇》：先生游南镇。一友指岩中花树问曰：“天下无心外之物。如此花树在深山中自开自落，于我心亦何相关？”先生曰：“<strong>你未看此花时，此花与汝心同归于寂</strong>。你来看此花时，则此花颜色一时明白起来。便知此花不在你的心外。”</li> <li>客观的事物没有被心知觉，就处于虚寂的状态（没有被照亮）。但我们看过之后，这朵花就进入了我们心中，具体的意义便不同了。当你骑着自行车时，你讨厌开车的人，当你开着车时，你讨厌骑自行车的人。<strong>同样一个场景，不同的角度不同的意义，所以善恶不在物本身，而在心内对物的投射</strong>。</li> <li>外在事物的意义是我们的内心赋予的，这个世界是我们内心的投射。佛说就是“相（好坏）由心生，境（客观存在）随心（思维定势）转”。我们的情绪不源于事情本身，而源于我们自己的想法。我们时时刻刻处理的都不是客观事物本身，而是主观想法。比如，无论选择哪一条路，我们总是美化我们没有走过的那条路，这就是遗憾，这个遗憾并不源于事情本身，而是源于我们自己的想法。</li> <li>人活在自己的认识之中，<strong>我们看的都是自己想看的，绝境不过是心境。所有的遇见都是恩典，都是照亮我们的内心世界，帮我们找到开关。</strong></li> <li>逆增上缘。</li> </ul> <blockquote> <p>[!tip] Reflections <strong>心外无物的意义在于看问题的角度</strong>。原本，我们认为我们是在观察世界，探究世界运行的规律，而心外无物的是视角的转置，向内去观察内心世界。如此，你把自己摆在了一个观察自己的角度，超脱了原来的情绪与应激反应，这就是明代的心理学。</p> </blockquote> <p>破山中贼易，破心中贼难。贼在心中，而不在心外，良知生而有之，无需外求。“致良知”，就是这样一个求知成圣的过程。</p> <ul> <li>”良知乃是天理昭明灵觉处“。儒家认为天赋良知，人人生来具有超越性，这是能辨善恶是非，洞察本质的能力，只不过被暂时遮蔽，需要求而得之。</li> <li>什么是良知？圣人的内在世界就是良知。儒学的根本目标是找到人人成圣的法门。心学提供了一个等价目标，就是人人都破心中之贼，人人皆于心中求得光明世界。王阳明遗言，”此心光明，亦复何言“，就是写照。</li> <li>致是动词，是获取与感悟的过程。生命情感的真相会在某些此刻不由自主的呈现（而不是理性的推理），此刻，我们听到了内心良知，就是感悟到了天理。理性思考不能感悟到良知，良知要在生命实践感悟到的。为什么是这样，知行合一。</li> </ul> <h2 id="知行合一">知行合一</h2> <blockquote> <p>[!info] English You do what you think, not what you think you should do.</p> </blockquote> <p>行先知后，还是知先行后，行易知难，还是知易行难（知道做不到）？都不是，王阳明总结了一个规律，“知行合一”:</p> <ul> <li><strong>常见误读是把知行合一理解为一种方法论，作为执行力的来源，认为道理知晓得透彻了，就可以有所行为了（知先行后），或作为认知方法，实践的多了就可以凝练真知（行先知后）。</strong> 实际上，知行合一不是一个目标，不存在做到做不到的问题，而是一个自然规律，“知是行之始，行是知之成”。<strong>知与行是一体两面的必然，没有先后，没有难易，不可分割，任何行为都有内在动机，所有动机都会有对应的外露（行为）。</strong></li> <li>但“知”，不是纯粹的知识，而是内心对世界的洞察，对事物本质的理解，对意义的把握。更高层次的知，是生命的动机，是佛家的佛性，道教的真心，儒家的良知，是圣人境界，是价值归属。行也不需要十分具体的行动，”一念发动处即是行“，一个念头，一种情绪即是行。知行合一的具体体现，比如，腹有诗书气自华，“知”会于举手投足之间自然流露。光明磊落，知行合一。</li> </ul> <blockquote> <p>[!example] <strong>“知而不行，并非真知”</strong></p> <ul> <li>很多人知道抽烟有害健康，但仍然抽烟，其实他们并不是真正知道抽烟有害健康，相反，他们的行为体现出他们内心仍然存在某些侥幸去合理化他们的行为，这代表了他们对这件事有不同理解（动机）。</li> </ul> </blockquote> <ul> <li>“有如知痛，必已自痛了，方知痛。知寒，必已自寒了。知饥，必已自饥了”，人时时刻刻都在知行的实践中，过程千姿百态。但如何学而致圣？王阳明总结就是“致良知”，它是心灵觉醒，体悟真知的过程。<strong>知行合一揭示这个过程不是一夜的顿悟，而是在生命实践中发掘人生意义，推广扩充天赋良知于万事万物。</strong> 外在有丰富的生命实践，内在有修筑一个光明世界，这个光明世界会照亮你的举手投足，你的举手投足又会为光明世界添砖加瓦。</li> <li>既然知行合一不存在做到做不到的问题，那么操作上，致良知是怎样开始的，那就是<strong>直接开始做，You are never be ready，在致知的实践中，世界在你的内心倒影，用你的内心用作一个观察器，去捕获天赋良知，感悟到生命情感的真相，去发掘意义，然后用继续用行动去追寻这种真相，去推广良知。</strong> Knowing yourself, your universe by doing, and observing what your are doing, you will know better about yourself, your universe in your mind. Because knowing and doing are essentially the same process, not two distinct or sequential ones.</li> <li>人生三大困惑：欲望，情绪，习性。知行合一可以帮助你克服习性问题。</li> </ul> <h2 id="相关观念">相关观念</h2> <h3 id="王德峰">王德峰</h3> <p>西方哲学研究的中心问题是一个人获得客观知识的方法是什么（如何滋养头脑）。中国哲学的本质是人生哲学，如何养心。</p> <p>陆王心学是宋明时期儒释道三者合流，是中国哲学的巅峰。（1）孟子，恻隐之心人皆有之；羞恶之心人皆有之；辞让之心人皆有之；是非之心人皆有之。万物皆备于我，反身而诚，乐莫大焉。（2）佛教，释迦牟礼是普通人，佛是已经觉悟了的众生，众生是尚未觉悟的佛。每个人都有佛性（即心即佛），因此要平等相待，不可轻视任何人。佛教认为人心受五蕴（色受想行识）遮蔽，消业方能成佛。</p> <h3 id="实践论">实践论</h3> <p>实践、认识、再实践、再认识，这种形式，循环往复以至无穷，而实践和认识之每一循环的内容，都比较地进到了高一级的程度。这就是辩证唯物论的全部认识论，这就是辩证唯物论的知行统一观。（毛泽东）</p> <h2 id="个人观点">个人观点</h2> <p>王阳明为学而至圣（儒家的至高人生目标）提供了一个等价命题，那就是视“内心光明”最高价值。这一目标有以下几个特点：</p> <ul> <li>关注内在世界，对外在世界纷扰不屑，“心外无物、心外无事、心外无理”。</li> <li>有人本主义心理学色彩。天赋良知体现了人的超越性，内心光明就是自我价值的实现，自我潜力的发掘。</li> </ul> <p>是唯心主义吗？是阿Q精神吗？王阳明自己可能不这么认为。知行合一是联系内在世界与生命实践的桥梁，有光明的内心世界，必然有惊世骇俗的生命实践。“知是行之始，行是知之成”，这就是实践论里的知识指导实践、实践检验真知，循环往复以至无穷的过程，只不过知行合一更强调实践与知识的统一性，即实践必然带来识，知必然所有实践。</p> <p>如果用贝叶斯认识论来理解，天理存在但是模糊（贝叶斯称不确定，儒家称遮蔽），所谓学而至圣，所谓内心光明，就是反复实践中求得一个极大的后验，足以洞悉至理，超凡入圣。</p> <h2 id="网络观念">网络观念</h2> <p><strong>“心外无物”是在理学“天父地母，民胞物与”的思想基础上推导出来的。</strong></p> <p>在“民胞物与”的世界里，宇宙是一个巨大的生命体，天地是我们父母，所有人是我的兄弟姐妹，万物是我的同类。既然宇宙是一个巨大的生命体，那么大至山河大地，小至草木鱼虫，远至日月星辰，近至我们自己，都是宇宙身上的某个器官或某个细胞，彼此都是一体相通的，这样就没有内外之别。</p> <p>张载说“为天地立心，为生民立命……”，为什么要给天地立心呀？</p> <p>“为天地立心”也是基于这套观念，天地本来没有心，但天地万物与我一体，人是万物之灵，即人心就是天地的心，“为天地立心”就是为人类自己立心。</p> <p>这个神秘的道理可以从朴素的观察得到证实：五谷和禽兽都可以成为人的食物，滋养人的生命，药石则可以治疗人的疾病，倘若彼此不是一体相通，滋养与治疗该如何可能呢？</p> <p>假如你接受这套观念，哪里还有内外，天地万物是我身体的延伸，我与天地一体，一切尽在心中。</p>]]></content><author><name>Zhihao</name></author><category term="notes"/><category term="人文"/><category term="人与自我"/><category term="lifebook"/><summary type="html"><![CDATA[心即理，心外无物，知行合一，致良知]]></summary></entry><entry xml:lang="eng"><title type="html">A tabular dataset ready for machine learning applications</title><link href="https://zhihaol.eu.org/blog/2023/subgrid/" rel="alternate" type="text/html" title="A tabular dataset ready for machine learning applications"/><published>2023-11-03T00:00:00+00:00</published><updated>2023-11-03T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/subgrid</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/subgrid/"><![CDATA[<p>On Zenodo, I have uploaded a dataset (DOI: 10.5281/zenodo.10048875) containing snow depth measurements retrieved from ICESat-2. In my previous work, I described the process as follows:</p> \[dh = IS2_{snow-free} - DEM\] <p>This formula represents the discrepancy between ICESat-2 and DEMs. To obtain precise snow depth measurements, the primary effort involves minimizing skewness and noise of \(dh\), as detailed in my previous post. Consequently, we derived snow depth from well-corrected DEMs, as available in the Zenodo dataset:</p> \[snow depth = IS2_{snow-on} - DEM - \hat{dh}\] <p>This dataset is a test run on snow depth retrieval workflow based on ICESat-2 (publication is under preparation).</p> <h2 id="advantages">Advantages</h2> <ul> <li>Open Access and Free: The ICESat-2 dataset is globally open-accessible and free of charge.</li> <li>Subgrid Variability: The dataset offers relatively high resolution, with an ideal footprint size of 13 - 17 m. It may be the only data source that contains subgrid variability of snow depth.</li> </ul> <h2 id="limitations">Limitations</h2> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/VArMIEU-480.webp 480w,https://i.imgur.com/VArMIEU-800.webp 800w,https://i.imgur.com/VArMIEU-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/VArMIEU.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Figure 1. The retrieved snow depth (2018.10-2022.10) in Hardangervidda each month. Much thick snow depth (&gt; 5 m) has been captured, demonstrating the 'high-resolution' of retrieved snow depth. However, it might still be less represented because of sparse sampling. </div> <ul> <li>Sparse Sampling: The sparse nature of the <a href="https://icesat-2.gsfc.nasa.gov/science/specs">sampling</a> makes it challenging to produce a completed snow depth map. In a year cycle, there is a 4 times repeat orbit with revisited tracks at 7.2 km and 14.4km away in different seasons. For this reason, we have to use a reference ground to bridge the gap between snow-on and snow-off segments to retrieve snow depth. The sparse sampling also leads to an unbalance of the dataset, making it more challenging to predict the correct distribution of the snow depth. Figure 1 shows the sampling in Hardangervidda each month, there are obvious gaps (15 km wide) during the snow season. Therefore we can expect more difficulty in this area.</li> <li>Sensitivity to bias correction. The accuracy of ICESat-2 snow depth is highly dependent on the bias correction process. <ul> <li>Vegetation introduces significant uncertainties in both ICESat-2 and DEMs.</li> <li>Permanent ice and inland water areas, which cannot be used as reference surfaces, must be excluded. However, many DEMs include seasonal snow patches, particularly in high elevations, which are not indicated by current permanent snow/ice masks. Similarly, lake masks may not be entirely suitable for all DEMs that are acquired in different seasons.</li> <li>In practice, many DEMs are old enough to accumulate many changes on the surface, such as landslides, erosion, and brushwood…All these require careful co-registration and bias correction.</li> </ul> </li> <li>ICESat-2 Ground-Finding Algorithm: <ul> <li>The parameterization of ICESat-2’s ground-finding algorithm significantly impacts retrieved snow depth. This algorithm uses a probability distribution function (PDF) of photons reflecting off the surface to determine exact elevations in the ATL08 product. If the algorithm or applied slope correction tends to capture or ignore specific signals, or if the footprint size averages out specific landforms, this can lead to systematic or scaling biases larger than the native footprint size. Under unideal conditions, the algorithm has produced lower estimation/interpolation, leading to negative skewness.</li> </ul> </li> <li>Lack of Validation: <ul> <li>Due to scaling and representativeness issues, it is challenging to compare the retrieved snow depth with ERA5 Land data (native 9km resolution, including permanent ice), or other gridded snow products.</li> <li>Large-scale snow surveys for validation are costly, making it difficult to validate the retrieved snow depth across various terrains.</li> </ul> </li> </ul> <h2 id="example-of-application">Example of Application</h2> <p>I am excited to unveil an application based on this dataset, focusing on extracting subgrid variability and its utilization in downscaling processes. The following is a flowchart illustrating this application:</p> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/KnfLKt1-480.webp 480w,https://i.imgur.com/KnfLKt1-800.webp 800w,https://i.imgur.com/KnfLKt1-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/KnfLKt1.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Flowchart illustrating the process of extracting snow depth from satellite altimetry data. The snow depth is derived using an elevation differencing workflow. Subsequently, a tree-structure-based regression model is trained and applied to predict the local variability of snow depth at any location and time. Satellite ICESat-2 graphics source: NASA.gov. (Zhihao, in preparation). </div> <h2 id="example-of-downscaled-snow-depth">Example of Downscaled Snow Depth</h2> <iframe width="560" height="315" src="https://www.youtube.com/embed/My1fSNGNxb4?si=U0VNY9c3lTXd9Zir" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen=""></iframe>]]></content><author><name>Zhihao</name></author><category term="notes"/><category term="DEM"/><category term="Snow"/><category term="ICESat-2"/><summary type="html"><![CDATA[Retrieved snow depth in Mainland Norway (2018.10-2022.10) based on ICESat-2 ATL 08 and DEMs]]></summary></entry><entry xml:lang="eng"><title type="html">ICESat-2 vs DTM1, DTM10, Copernicus30, FABDEM</title><link href="https://zhihaol.eu.org/blog/2023/dataset/" rel="alternate" type="text/html" title="ICESat-2 vs DTM1, DTM10, Copernicus30, FABDEM"/><published>2023-11-01T00:00:00+00:00</published><updated>2023-11-01T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/dataset</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/dataset/"><![CDATA[<p>What is the difference between DTM1, DTM10, Copernicus GLO30, and FABDEM? How do they compare with ICESat-2? On Zenodo, I uploaded a dataset (10.5281/zenodo.10048875) to answer these questions.</p> <p>The first step to compare different elevation products is co-registration. Here, I used the Gradient Descent Co-registration method that I mentioned in a previous post.</p> <h4 id="the-performance-of-co-registration">The Performance of Co-registration</h4> <p>After co‑registration, the percentage of points with a deviation less than 0.5 m from ICESat‑2 snow‑free segments is calculated to be 48.9%, 49.13%, 23.7%, and 27.2% for DTM1, DTM10, GLO30, and FABDEM respectively. In terms of NMAD (Normalized Median Absolute Deviation), for example, the improvement for DTM10 is from 0.99m to 0.77m (NMAD). If we attribute the residual bias partially to ICESat-2, we can use the subset_te_flag to exclude bad segments. This will result in better metrics for the co-registered DEM (Table 4.2 in my thesis).</p> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/hPl9Hdw-480.webp 480w,https://i.imgur.com/hPl9Hdw-800.webp 800w,https://i.imgur.com/hPl9Hdw-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/hPl9Hdw.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/YTYUbHN-480.webp 480w,https://i.imgur.com/YTYUbHN-800.webp 800w,https://i.imgur.com/YTYUbHN-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/YTYUbHN.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Figure 1.: Statistic Binning Analysis: Co-registration and Bias Correction (ICESat-2 - DTM10). The dataset was divided into categories at three stages, and for each category, a violin plot displays the distribution of data. The median of each dataset is represented by a white dot at the center of the violin. The thick bar illustrates the 25% and 75% quantiles, respectively. If the violin is skewed to one side, it indicates a deviation from normal distribution. The aspect-dependent bias is identified and eliminated by co-registration (a); The negative bias is widespread after co-registration (a, b, c). </div> \[dh = IS2_{snow-free} - DEM\] <p>From figure 1, we can clearly see the aspect-dependent error, the fingerprint of the geo-referencing mismatch, is removed through co-registration, as indicated by the median value (Q50) being located close to zero for all bins (a). Although ATL08 (version 5) has contained geolocation errors since 2021.12 (which have been fixed in version 6), the overall amount of data from that period is not significant to produce wrong co-registration. Additionally, gradient descent co-registration has a certain ability to suppress noise.</p> <p>The figure also shows significant residual bias after co-registration. The dh has a nearly linear relationship with slope and curvature (b, c). This bias requires further correction.</p> <h3 id="the-difference-after-bias-correction">The Difference after Bias Correction</h3> <p>After the bias correction, the comparison between DTM1 and DTM10 indicates that they both achieved a high level of accuracy on low slope categories, with less than 1 m and 0.5 m NMAD in 65% and 35% of cases, respectively. In contrast, the worst results are observed in the top 5% of <em>slope</em> quantiles and the least 5% of <em>n_photons</em> quantiles, which resulted in an NMAD of 2.4 m and 1.8 m, respectively. The bias correction applied to GLO30 produced better results than for FAB, with approximately 40% of measurements showing an NMAD of less than 1 m on the low slopes category (Figure 4.3 in my thesis). These improvements demonstrate the effectiveness of our approach in eliminating biases.</p> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/ShRB4wX-480.webp 480w,https://i.imgur.com/ShRB4wX-800.webp 800w,https://i.imgur.com/ShRB4wX-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/ShRB4wX.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Figure 2.: The Overall Distribution of DEM errors. DTM10 (upper left), DTM1 (upper right), COP30 (lower left), FAB (lower right). The statistical data for raw, after co-registration, and bias-correction are represented by blue, black, and red colors respectively. The 25% quantile (Q1) and the 75% quantile (Q3) are displayed from left to right along the x-axis. After bias correction, Q1 has moved closer to the median, indicating a reduction in negative skewness. Please note that a window of -10 m to 10 m was used to filter outliers in the metrics employed. And, the number of measurements is given by n (DEM) and N (total). </div> <p>Figure 2 shows the overall $dh$ distributions, which exhibit a negative skewness with long tails on the left side. Prior to bias correction, GLO30 had a median of -1.30 m, while FAB, which is a version of GLO30 with vegetation and building removed, exhibited less skewness. Although DTM1 and DTM10 had similar shapes, their 25% quantiles (Q1) suggested the presence of systematic biases. After implementing the bias correction, there was a noticeable enhancement in the overall symmetry observed across all four scenarios. This was supported by the fact that Q1 moved nearer to the median, while Q3 remained relatively stable. Ultimately, there is an overall NMAD of 0.56 m and 0.57 m for DTM10 and DTM1, respectively. GLO30 and FAB had higher NMAD of 0.98 m and 1.08 m. It is evident that bias correction led to a significant improvement in the accuracy of the results.</p> <p>One meter of NMAD – This might be the highest precision that could be achieved for Copernicus GLO30 and FAB. On the other hand, we noticed negative skewness exhibited on all DEMs for unknown reason.</p> <h4 id="negative-skewness-of-icesat-2">Negative Skewness of ICESat-2</h4> <p>The negative values have a spatial distribution pattern, which is widely spread on convex terrain. The figure below plots nine curvature combinations categorized by profile curvature and plan curvature. The aggregated mean value of elevation differences reveals that convex ridged terrain (positive plan curvature and negative profile curvature) typically exhibits negative skewness (a,g). A concave bowl-shaped terrain with negative profile curvature does not have much negative skewness, even at high slopes (b), thereby indicating that the observed linear relationship between bias and slope is mainly contributed to by plan curvature.</p> <p>Naturally, there is an overestimation over concave, but underestimation of convex because of the limitation of coarse resolution (or footprint). However, this should be symmetrical bias and not as significant as what we observed in terms of negative skewness (Figure 1 b,c).</p> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/WHUrfZ0-480.webp 480w,https://i.imgur.com/WHUrfZ0-800.webp 800w,https://i.imgur.com/WHUrfZ0-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/WHUrfZ0.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Figure 3.: Skewness Variations in Convex and Concave Terrains. The elevation difference is between ICESat-2 snow free segments and DTM1. The color points indicate the aggregated mean value of the elevation difference. Plots (a), (b) and (c) depict the bias pattern along plan curvature, profile curvature, slope and canopy height before the bias correction. A triangle window (a) indicating a bias free condition, the rest of area either gives negative bias (blue) or positive bias (red). After the bias correction, most of the negative bias is removed (d, e, f). Plot (g) shows a schematic of curvature combinations. The plot reveals that there is a negative correlation between profile curvature and plan curvature. </div> <h4 id="using-h_te_mean-h_te_best_fit-or-h_te_best_fit_geosegments">Using h_te_mean, h_te_best_fit or h_te_best_fit_geosegments?</h4> <p>One concern is what the real resolution of the ICESat-2 ATL08 product is. Which elevation should I use? The real resolution of the ICESat-2 ATL08 product is not 100 meters. 100 m of segment is just how the dataset is structured, considering easy delivery of the canopy information in segments. There are several elevations provided by ATL08:</p> <ul> <li>h_te_mean: The real mean elevation of the segment. I have tried aggregating DTM1 by a certain window size and comparing the aggregated mean elevation with h_te_mean. However, there is no fixed window that is equivalent to the footprint of a segment (not 100 m or any other fixed value). The photons may not be evenly spread over either a 100 m long segment window or any other size.</li> <li>h_te_interp: h_te_interp is an interpolated value, which is generally worse than h_te_best_fit in many cases.</li> <li>h_te_best_fit: Good to use; however, when the geosegment has missing, it is very important to use subset_te_flag to exclude bad measurements. When five geosegments are available, the quality significantly improves.</li> <li>h_te_best_fit_20m_2: The best fit elevation at the midpoint of the segment.</li> </ul> <p>In the beginning, I used h_te_best_fit; however, when the midpoint is not available, this value becomes an interpolated value (not sure if it equals h_te_interp). And this ‘interpolated value’ contains bias (generally tending to underestimate surface height). Therefore, using h_te_best_fit_20m_2 as the elevation of the midpoint of a segment eliminates any worry about interpolated values.</p> <h3 id="dem-bias-correction-model-based-on-icesat-2-snow-off-measurements">DEM Bias Correction Model Based on ICESat-2 Snow-off Measurements</h3> <div class="row">     <div class="col-sm mt-3 mt-md-0">     <figure> <picture> <source class="responsive-img-srcset" srcset="https://i.imgur.com/vpZ6BMg-480.webp 480w,https://i.imgur.com/vpZ6BMg-800.webp 800w,https://i.imgur.com/vpZ6BMg-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="https://i.imgur.com/vpZ6BMg.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure>     </div> </div> <div class="caption"> Figure 4.: Model interpretation. The bias correction model of DTM1 - ICESat-2 (a) The bias correction model of COP30 - ICESat-2 (b). The interaction value between slope and plan curvature (c). The subset flag and difference. </div> <p>As mentioned, the difference between ICESat-2 and DEMs has been corrected by a model. What does the model tell us? I used SHAP to interpret the model’s behavior.</p> <p>The features used in the model are listed from the most influential on the top to the least influential on the bottom. A negative SHAP value means a negative bias (ICESat-2 - DEMs &lt; 0), and vice versa. The feature value has been normalized to show positive or negative correlation. In general, most of the bias is negative. For instance, Figure 4 a shows the model using h_te_best_fit_20m_2. Here, the difference is defined as h_te_best_fit_20m_2 - h_te_best_fit. A positive difference (when h_te_best_fit_20m_2 &gt; h_te_best_fit) normally contributes to a negative bias, which means h_te_best_fit has an even more severe negative skewness. This situation often occurs when only 2 to 4 geosegments are available (d). Other features that lead to negative bias include steep slope, convex terrain, canopy, and low standard deviation of ground photons, but positive skewness of ground photons. Additionally, TPI in a 270-meter scale is the least feature, meaning that the elevation difference is purely a micro topographic problem. For the model of COP30 - ICESat-2, the elevation difference is very related to vegetation coverage, canopy and TPI in 90 meters (3 pixel on COP30). <strong>As the TPI behavior opposite with previous model, statistically, ICESat-2 has a better resolution than COP30.</strong></p> <p>As demonstrated in figure 3, in figure 4 c, the behavior of negative bias is totally opposite on convex terrain (plan curvature &gt; 0) and concave terrain (plan curvature &lt; 0), which means that currently most of slope-only-dependent bias correction is not exactly correct.</p> <p>Overall, the interpolation and slope correction algorithm of ICESat-2 ALT08 is quite interesting. However, it tends to underestimate surface height when there are no photons at the midpoint. Where h_te_best_fit_20m_2 is available, this algorithm may mistakenly consider it as part of the canopy and remove it. This tendency introduces residual bias, result in such as negative snow depth.</p> <h4 id="is-it-possible-using-negative-snow-depth-in-dem-bias-correction">Is it Possible Using Negative Snow Depth in DEM Bias Correction?</h4> <p>Naturally, the distribution of dh is normal and symmetrical. When we are doing elevation differencing for snow depth, we also obtain snow depth with uncertainty in both tails. However, as the snow cover is very small relative to the uncertainties of DEMs-ICESat-2 (0.5m to 1m), many snow depth measurements (nearly 30%) result in negative values. Excluding these negative snow depths interferes significantly with small value observations. One idea is to use these snow-on measurements to further correct dh in an iterative process. It is very risky because such corrections are applied to specific terrains.</p>]]></content><author><name>Zhihao</name></author><category term="notes"/><category term="DEM"/><category term="Snow"/><category term="ICESat-2"/><summary type="html"><![CDATA[Dataset is available on Zenodo]]></summary></entry><entry xml:lang="eng"><title type="html">My thesis in a page</title><link href="https://zhihaol.eu.org/blog/2023/abstract/" rel="alternate" type="text/html" title="My thesis in a page"/><published>2023-05-12T00:00:00+00:00</published><updated>2023-05-12T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/abstract</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/abstract/"><![CDATA[]]></content><author><name>Zhihao</name></author><category term="climate"/><category term="geomatics"/><category term="remote_sensing"/><category term="machine_learning"/><category term="links"/><category term="links"/><summary type="html"><![CDATA[Advancements in Snow Depth Retrieval Using Satellite Altimetry and Machine Learning]]></summary></entry><entry xml:lang="eng"><title type="html">My Poster at LATICE 2023</title><link href="https://zhihaol.eu.org/blog/2023/LATICE2023/" rel="alternate" type="text/html" title="My Poster at LATICE 2023"/><published>2023-03-26T00:00:00+00:00</published><updated>2023-03-26T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/LATICE2023</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/LATICE2023/"><![CDATA[]]></content><author><name>Zhihao</name></author><category term="climate"/><category term="datascience"/><category term="links"/><category term="links"/><summary type="html"><![CDATA[Snow Depth from Satellite Laser Altimetry - Co-registration, Bias Correction, and Statistical Downscaling]]></summary></entry><entry xml:lang="eng"><title type="html">My talk at SDG 2023</title><link href="https://zhihaol.eu.org/blog/2023/SDG2023/" rel="alternate" type="text/html" title="My talk at SDG 2023"/><published>2023-02-07T00:00:00+00:00</published><updated>2023-02-07T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/SDG2023</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/SDG2023/"><![CDATA[]]></content><author><name>Zhihao</name></author><category term="sustainable"/><category term="climate"/><category term="links"/><category term="links"/><summary type="html"><![CDATA[The scerets behind the snow depth]]></summary></entry><entry xml:lang="eng"><title type="html">DEM Coreg is a bound-restricted minimizing problem</title><link href="https://zhihaol.eu.org/blog/2023/dem-coreg/" rel="alternate" type="text/html" title="DEM Coreg is a bound-restricted minimizing problem"/><published>2023-01-02T00:00:00+00:00</published><updated>2023-01-02T00:00:00+00:00</updated><id>https://zhihaol.eu.org/blog/2023/dem-coreg</id><content type="html" xml:base="https://zhihaol.eu.org/blog/2023/dem-coreg/"><![CDATA[<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">importlib</span> <span class="kn">import</span> <span class="nb">reload</span>
<span class="kn">import</span> <span class="n">pandas</span> <span class="k">as</span> <span class="n">pd</span>
<span class="kn">import</span> <span class="n">xdem</span>
<span class="kn">import</span> <span class="n">xsnow</span>
<span class="kn">from</span> <span class="n">xsnow.godh</span> <span class="kn">import</span> <span class="n">load_gdf</span><span class="p">,</span><span class="n">best_shift_px</span><span class="p">,</span><span class="n">get_dh_by_shift_px_gdf</span><span class="p">,</span><span class="n">load_gdf</span>
<span class="kn">from</span> <span class="n">xsnow.misc</span> <span class="kn">import</span> <span class="n">df_sampling_from_dem</span>
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">reload</span><span class="p">(</span><span class="n">xsnow</span><span class="p">.</span><span class="n">misc</span><span class="p">)</span>
<span class="nf">reload</span><span class="p">(</span><span class="n">xsnow</span><span class="p">.</span><span class="n">godh</span><span class="p">)</span>
<span class="nf">reload</span><span class="p">(</span><span class="n">xdem</span><span class="p">)</span>
<span class="nf">reload</span><span class="p">(</span><span class="n">xdem</span><span class="p">.</span><span class="n">coreg</span><span class="p">)</span>
<span class="kn">from</span> <span class="n">importlib</span> <span class="kn">import</span> <span class="nb">reload</span>
<span class="kn">import</span> <span class="n">xsnow</span>
<span class="kn">import</span> <span class="n">xsnow.godh</span>
</code></pre></div></div> <h2 id="dem-coreg-is-a-bound-restricted-minimizing-problem">DEM Coreg is a Bound-restricted Minimizing Problem.</h2> <ul> <li>DEM Coreg is a bound-restricted minimizing problem, but suffering from noise.</li> </ul> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Loading the dataset.
</span>
<span class="c1"># 1 ICESat-2 snow free measurements.
</span><span class="n">sf_gdf</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="nf">read_csv</span><span class="p">(</span><span class="sh">'</span><span class="s">snow_free_land_gdf_normal_qc.csv</span><span class="sh">'</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="sh">'</span><span class="se">\t</span><span class="sh">'</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="sh">'</span><span class="s">utf-8</span><span class="sh">'</span><span class="p">)</span>

<span class="c1"># 2 DTM 10 exmaple (10 m Norway DTM-10)
#fid_dtm10 = r'\\hypatia.uio.no\lh-mn-geofag-felles\projects\snowdepth\zhihaol\data\dtm10_finse_merged.tif'
</span><span class="n">fid_dtm10</span> <span class="o">=</span> <span class="sa">r</span><span class="sh">'</span><span class="s">\\hypatia.uio.no\lh-mn-geofag-felles\projects\snowdepth\data\DEM\Norway\DTM10_UTM33\6702_4_10m_z33.tif</span><span class="sh">'</span>

<span class="n">dtm_10_</span> <span class="o">=</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="n">fid_dtm10</span><span class="p">)</span>
<span class="n">dtm_10</span> <span class="o">=</span> <span class="n">dtm_10_</span> <span class="o">+</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="sh">'</span><span class="s">no_kv_HREF2018B_NN2000_EUREF89.tif</span><span class="sh">'</span><span class="p">).</span><span class="nf">reproject</span><span class="p">(</span><span class="n">dtm_10_</span><span class="p">,</span><span class="n">resampling</span><span class="o">=</span><span class="sh">'</span><span class="s">bilinear</span><span class="sh">'</span><span class="p">)</span>

<span class="c1"># 3 DTM 1 exmaple (1 m Norway DTM-1)
</span><span class="n">fid_dtm1</span> <span class="o">=</span> <span class="sa">r</span><span class="sh">'</span><span class="s">\\hypatia.uio.no\lh-mn-geofag-felles\projects\snowdepth\data\DEM\Norway\DTM1_UTM33\DTM1_11-11_UTM33\33-113-119.tif</span><span class="sh">'</span>
<span class="n">dtm_1_</span> <span class="o">=</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="n">fid_dtm1</span><span class="p">)</span>
<span class="n">dtm_1</span> <span class="o">=</span> <span class="n">dtm_1_</span> <span class="o">+</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="sh">'</span><span class="s">no_kv_HREF2018B_NN2000_EUREF89.tif</span><span class="sh">'</span><span class="p">).</span><span class="nf">reproject</span><span class="p">(</span><span class="n">dtm_1_</span><span class="p">,</span><span class="n">resampling</span><span class="o">=</span><span class="sh">'</span><span class="s">bilinear</span><span class="sh">'</span><span class="p">)</span>

<span class="c1"># subset snow free measurements
</span><span class="kn">from</span> <span class="n">xsnow.godh</span> <span class="kn">import</span> <span class="n">load_gdf</span>
<span class="n">sf_subset_dtm10_</span> <span class="o">=</span> <span class="nf">load_gdf</span><span class="p">(</span><span class="n">sf_gdf</span><span class="p">,</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">h_te_best_fit</span><span class="sh">'</span><span class="p">)</span>
<span class="n">sf_subset_dtm1_</span> <span class="o">=</span> <span class="nf">load_gdf</span><span class="p">(</span><span class="n">sf_gdf</span><span class="p">,</span><span class="n">dtm_1</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">h_te_best_fit</span><span class="sh">'</span><span class="p">)</span>

<span class="nf">print</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">The length of sf_subset_dtm10: </span><span class="si">{</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_subset_dtm10_</span><span class="p">)</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">)</span>
<span class="nf">print</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">The length of sf_subset_dtm1: </span><span class="si">{</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_subset_dtm1_</span><span class="p">)</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">)</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>The length of sf_subset_dtm10: 44149 points
The length of sf_subset_dtm1: 1586 points
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">itertools</span> <span class="kn">import</span> <span class="n">product</span>
<span class="kn">import</span> <span class="n">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="kn">import</span> <span class="n">matplotlib.pyplot</span> <span class="k">as</span> <span class="n">plt</span>
<span class="kn">import</span> <span class="n">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="kn">import</span> <span class="n">xdem</span>
<span class="kn">from</span> <span class="n">xsnow.misc</span> <span class="kn">import</span> <span class="n">df_sampling_from_dem</span>

<span class="k">def</span> <span class="nf">yield_surface_noise</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_gdf_subset</span><span class="p">,</span>
                        <span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">stat</span><span class="o">=</span><span class="sh">'</span><span class="s">nmad</span><span class="sh">'</span><span class="p">,</span> <span class="n">number_corg</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">downsampling_list</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">h_te_best_fit</span><span class="sh">'</span><span class="p">,</span>
                        <span class="n">nuthkaab</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span><span class="n">gdc_qc</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span><span class="n">weight</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
    
    <span class="k">if</span> <span class="n">number_corg</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
        <span class="n">number_corg</span> <span class="o">=</span> <span class="mi">1</span>
        
    <span class="k">if</span> <span class="n">downsampling_list</span><span class="p">:</span>
        <span class="k">for</span> <span class="n">sampling</span> <span class="ow">in</span> <span class="n">downsampling_list</span><span class="p">:</span>
            <span class="n">res</span> <span class="o">=</span> <span class="p">[</span><span class="nf">best_shift_px</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span> <span class="n">sf_gdf_subset</span><span class="p">,</span> <span class="n">disp</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span><span class="n">stat</span><span class="o">=</span><span class="n">stat</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="n">perc_t</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="n">std_t</span><span class="p">,</span> <span class="n">downsampling</span><span class="o">=</span><span class="n">sampling</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="n">z_name</span><span class="p">,</span><span class="n">weight</span><span class="o">=</span><span class="n">weight</span><span class="p">)</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">number_corg</span><span class="p">)]</span>
            <span class="n">nmad</span><span class="p">,</span><span class="n">rmse</span><span class="p">,</span><span class="n">std</span><span class="p">,</span><span class="n">median</span> <span class="o">=</span> <span class="nf">surface_noise</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_gdf_subset</span><span class="p">,</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="n">perc_t</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="n">std_t</span><span class="p">,</span><span class="n">downsampling</span><span class="o">=</span><span class="n">sampling</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="n">z_name</span><span class="p">)</span>
            
            <span class="k">if</span> <span class="n">nuthkaab</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
                <span class="k">try</span><span class="p">:</span>
                    <span class="n">func</span> <span class="o">=</span> <span class="n">xdem</span><span class="p">.</span><span class="n">coreg</span><span class="p">.</span><span class="nc">NuthKaab</span><span class="p">()</span>
                    <span class="n">func</span><span class="p">.</span><span class="nf">fit_pts</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">.</span><span class="nf">sample</span><span class="p">(</span><span class="n">frac</span><span class="o">=</span><span class="n">sampling</span><span class="o">/</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">),</span><span class="n">random_state</span><span class="o">=</span><span class="mi">42</span><span class="p">),</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">mask_highcurv</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
                    <span class="n">res_nk</span> <span class="o">=</span> <span class="p">(</span><span class="n">func</span><span class="p">.</span><span class="n">_meta</span><span class="p">[</span><span class="sh">"</span><span class="s">offset_east_px</span><span class="sh">"</span><span class="p">],</span><span class="n">func</span><span class="p">.</span><span class="n">_meta</span><span class="p">[</span><span class="sh">"</span><span class="s">offset_north_px</span><span class="sh">"</span><span class="p">],</span><span class="n">func</span><span class="p">.</span><span class="n">_meta</span><span class="p">[</span><span class="sh">"</span><span class="s">bias</span><span class="sh">"</span><span class="p">],</span><span class="n">func</span><span class="p">.</span><span class="n">_meta</span><span class="p">[</span><span class="sh">"</span><span class="s">nmad</span><span class="sh">"</span><span class="p">])</span>
                    <span class="nf">print</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">NuthKaab coreg on </span><span class="si">{</span><span class="n">sampling</span><span class="si">}</span><span class="s">: </span><span class="si">{</span><span class="n">res_nk</span><span class="si">}</span><span class="sh">'</span><span class="p">)</span>
                <span class="k">except</span> <span class="nb">ValueError</span><span class="p">:</span>
                    <span class="n">res_nk</span> <span class="o">=</span> <span class="bp">None</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="n">res_nk</span> <span class="o">=</span> <span class="bp">None</span>

            <span class="k">if</span> <span class="n">gdc_qc</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
                <span class="n">sf_subset_dtm10_ref</span> <span class="o">=</span> <span class="n">sf_gdf_subset</span><span class="p">.</span><span class="nf">query</span><span class="p">(</span><span class="sh">'</span><span class="s">subset_te_flag == 5</span><span class="sh">'</span><span class="p">)</span>
                <span class="n">res_qc</span> <span class="o">=</span> <span class="nf">best_shift_px</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span> <span class="n">sf_subset_dtm10_ref</span><span class="p">,</span> <span class="n">disp</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span><span class="n">stat</span><span class="o">=</span><span class="n">stat</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="n">perc_t</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="n">std_t</span><span class="p">,</span> <span class="n">downsampling</span><span class="o">=</span><span class="n">sampling</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="n">z_name</span><span class="p">)</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="n">res_qc</span> <span class="o">=</span> <span class="bp">None</span>

            <span class="k">yield</span> <span class="n">nmad</span><span class="p">,</span> <span class="n">rmse</span><span class="p">,</span> <span class="n">std</span><span class="p">,</span> <span class="n">median</span><span class="p">,</span><span class="n">res</span><span class="p">,</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_qc</span><span class="p">,</span><span class="nf">min</span><span class="p">(</span><span class="n">sampling</span><span class="p">,</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">surface_noise</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_gdf_subset</span><span class="p">,</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">downsampling</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">h_te_best_fit</span><span class="sh">'</span><span class="p">):</span>

    <span class="k">if</span> <span class="nf">isinstance</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">,</span><span class="n">xdem</span><span class="p">.</span><span class="n">DEM</span><span class="p">):</span>
        <span class="n">sf_gdf_subset</span> <span class="o">=</span> <span class="nf">df_sampling_from_dem</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_gdf_subset</span><span class="p">,</span><span class="n">samples</span><span class="o">=</span><span class="mi">10000</span><span class="p">)</span>

    <span class="k">if</span> <span class="n">downsampling</span> <span class="ow">and</span> <span class="nf">len</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">)</span> <span class="o">&gt;</span> <span class="n">downsampling</span><span class="p">:</span>
        <span class="n">sf_gdf_subset</span> <span class="o">=</span> <span class="n">sf_gdf_subset</span><span class="p">.</span><span class="nf">sample</span><span class="p">(</span><span class="n">frac</span><span class="o">=</span><span class="n">downsampling</span><span class="o">/</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">),</span><span class="n">random_state</span><span class="o">=</span><span class="mi">42</span><span class="p">).</span><span class="nf">copy</span><span class="p">()</span>
        <span class="nf">print</span><span class="p">(</span><span class="sh">'</span><span class="s">Running on downsampling. The length of the gdf:</span><span class="sh">'</span><span class="p">,</span><span class="nf">len</span><span class="p">(</span><span class="n">sf_gdf_subset</span><span class="p">))</span>
        <span class="nf">print</span><span class="p">(</span><span class="sh">'</span><span class="s">Set downsampling = other value or False to make a change.</span><span class="sh">'</span><span class="p">)</span>

    <span class="n">bond_</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">arange</span><span class="p">(</span><span class="n">bond</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">bond</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span><span class="n">bond</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
    <span class="n">nmad</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">zeros</span><span class="p">((</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">),</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">)))</span>
    <span class="n">rmse</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">zeros</span><span class="p">((</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">),</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">)))</span>
    <span class="n">std</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">zeros</span><span class="p">((</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">),</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">)))</span>
    <span class="n">median</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">zeros</span><span class="p">((</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">),</span><span class="nf">len</span><span class="p">(</span><span class="n">bond_</span><span class="p">)))</span>

    <span class="k">for</span> <span class="n">e</span><span class="p">,</span><span class="n">n</span> <span class="ow">in</span> <span class="nf">product</span><span class="p">(</span><span class="n">bond_</span><span class="p">,</span><span class="n">bond_</span><span class="p">):</span>
        <span class="n">results</span> <span class="o">=</span> <span class="nf">get_dh_by_shift_px_gdf</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_gdf_subset</span><span class="p">,(</span><span class="n">e</span><span class="p">,</span><span class="n">n</span><span class="p">),</span><span class="mi">0</span><span class="p">,</span><span class="n">stat</span><span class="o">=</span><span class="sh">'</span><span class="s">all</span><span class="sh">'</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="n">perc_t</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="n">std_t</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="n">z_name</span><span class="p">)</span>
        <span class="n">row</span> <span class="o">=</span> <span class="nf">int</span><span class="p">((</span><span class="n">n</span><span class="o">-</span><span class="n">bond</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span><span class="o">/</span><span class="n">bond</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
        <span class="n">col</span> <span class="o">=</span> <span class="nf">int</span><span class="p">((</span><span class="n">e</span><span class="o">-</span><span class="n">bond</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span><span class="o">/</span><span class="n">bond</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
        <span class="n">nmad</span><span class="p">[</span><span class="n">row</span><span class="p">,</span><span class="n">col</span><span class="p">]</span> <span class="o">=</span> <span class="n">results</span><span class="p">[</span><span class="sh">'</span><span class="s">nmad</span><span class="sh">'</span><span class="p">]</span>
        <span class="n">rmse</span><span class="p">[</span><span class="n">row</span><span class="p">,</span><span class="n">col</span><span class="p">]</span> <span class="o">=</span> <span class="n">results</span><span class="p">[</span><span class="sh">'</span><span class="s">rmse</span><span class="sh">'</span><span class="p">]</span>
        <span class="n">std</span><span class="p">[</span><span class="n">row</span><span class="p">,</span><span class="n">col</span><span class="p">]</span> <span class="o">=</span> <span class="n">results</span><span class="p">[</span><span class="sh">'</span><span class="s">std</span><span class="sh">'</span><span class="p">]</span>
        <span class="n">median</span><span class="p">[</span><span class="n">row</span><span class="p">,</span><span class="n">col</span><span class="p">]</span> <span class="o">=</span> <span class="n">results</span><span class="p">[</span><span class="sh">'</span><span class="s">median</span><span class="sh">'</span><span class="p">]</span>

    <span class="k">return</span> <span class="n">nmad</span><span class="p">,</span><span class="n">rmse</span><span class="p">,</span><span class="n">std</span><span class="p">,</span><span class="n">median</span>

<span class="k">def</span> <span class="nf">surface_plot</span><span class="p">(</span><span class="n">Z</span><span class="p">,</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">2</span><span class="p">),</span>
                 <span class="n">ax</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span>
                 <span class="n">res</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">res_3</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">minimal</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span><span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">[m]</span><span class="sh">'</span><span class="p">,</span>
                 <span class="n">z_step_level</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">):</span>

    <span class="k">if</span> <span class="n">ax</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
        <span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="p">.</span><span class="nf">subplots</span><span class="p">(</span><span class="n">subplot_kw</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">projection</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">3d</span><span class="sh">"</span><span class="p">})</span>
    <span class="k">if</span> <span class="n">xlim</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
        <span class="n">xlim</span><span class="o">=</span><span class="n">bond</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span>
    <span class="k">if</span> <span class="n">ylim</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
        <span class="n">ylim</span><span class="o">=</span><span class="n">bond</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span>

    <span class="c1"># Make a grid.
</span>    <span class="n">X</span> <span class="o">=</span> <span class="n">Y</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">arange</span><span class="p">(</span><span class="n">bond</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">bond</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span><span class="n">bond</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
    <span class="n">X</span><span class="p">,</span> <span class="n">Y</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">meshgrid</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">)</span>

    <span class="c1"># plot minimal
</span>    <span class="n">zmin</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">min</span><span class="p">(</span><span class="n">Z</span><span class="p">)</span>
    <span class="n">zmax</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">max</span><span class="p">(</span><span class="n">Z</span><span class="p">)</span>
    <span class="n">mask</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">Z</span><span class="p">)</span> <span class="o">==</span> <span class="n">zmin</span>
    <span class="c1">#color = np.where(mask,'blue','red')
</span>    
    <span class="c1"># Plot the 3D surface
</span>    <span class="n">ax</span><span class="p">.</span><span class="nf">plot_surface</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="n">edgecolor</span><span class="o">=</span><span class="sh">'</span><span class="s">royalblue</span><span class="sh">'</span><span class="p">,</span> <span class="n">lw</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">rstride</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">cstride</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span>
                    <span class="n">alpha</span><span class="o">=</span><span class="mf">0.3</span><span class="p">)</span>

    <span class="c1"># Plot projections of the contours for each dimension.  By choosing offsets
</span>    <span class="c1"># that match the appropriate axes limits, the projected contours will sit on
</span>    <span class="c1"># the 'walls' of the graph.
</span>    <span class="n">kwargs</span> <span class="o">=</span> <span class="p">{}</span>
    <span class="k">if</span> <span class="n">z_step_level</span><span class="p">:</span>
        <span class="n">kwargs</span><span class="p">[</span><span class="sh">'</span><span class="s">levels</span><span class="sh">'</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">arange</span><span class="p">(</span><span class="n">zmin</span><span class="p">,</span> <span class="n">zmax</span> <span class="o">+</span> <span class="n">z_step_level</span><span class="p">,</span> <span class="n">z_step_level</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">z_step_level</span> <span class="o">=</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">zlim_s</span>
        <span class="n">kwargs</span><span class="p">[</span><span class="sh">'</span><span class="s">levels</span><span class="sh">'</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">arange</span><span class="p">(</span><span class="n">zmin</span><span class="p">,</span> <span class="n">zmax</span> <span class="o">+</span> <span class="n">z_step_level</span><span class="p">,</span> <span class="n">z_step_level</span><span class="p">)</span>


    <span class="n">z_contour</span> <span class="o">=</span> <span class="n">ax</span><span class="p">.</span><span class="nf">contour</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span><span class="n">zdir</span><span class="o">=</span><span class="sh">'</span><span class="s">z</span><span class="sh">'</span><span class="p">,</span> <span class="n">offset</span><span class="o">=</span><span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">cmap</span><span class="o">=</span><span class="sh">'</span><span class="s">coolwarm</span><span class="sh">'</span><span class="p">,</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
    <span class="n">ax</span><span class="p">.</span><span class="nf">contour</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="n">zdir</span><span class="o">=</span><span class="sh">'</span><span class="s">x</span><span class="sh">'</span><span class="p">,</span> <span class="n">offset</span><span class="o">=</span><span class="n">xlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">cmap</span><span class="o">=</span><span class="sh">'</span><span class="s">coolwarm</span><span class="sh">'</span><span class="p">)</span>
    <span class="n">ax</span><span class="p">.</span><span class="nf">contour</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="n">zdir</span><span class="o">=</span><span class="sh">'</span><span class="s">y</span><span class="sh">'</span><span class="p">,</span> <span class="n">offset</span><span class="o">=</span><span class="n">ylim</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">cmap</span><span class="o">=</span><span class="sh">'</span><span class="s">coolwarm_r</span><span class="sh">'</span><span class="p">)</span>
    
    <span class="c1">#ax.clabel(z_contour, kwargs['levels'][0::2], z_contour.levels[::2], inline=True,fmt ='%1.1f', colors='black', fontsize='x-large')
</span>
    <span class="k">if</span> <span class="n">minimal</span><span class="p">:</span>
        <span class="c1"># scatter minimal on bottom
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">mask</span><span class="p">],</span> <span class="n">Y</span><span class="p">[</span><span class="n">mask</span><span class="p">],</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">blue</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">minimal</span><span class="sh">'</span><span class="p">)</span>
        <span class="c1">#scatter minimal on z-axis
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">xlim</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">ylim</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">zmin</span><span class="p">,</span><span class="n">marker</span><span class="o">=</span><span class="sh">"</span><span class="s">x</span><span class="sh">"</span><span class="p">,</span><span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">blue</span><span class="sh">'</span><span class="p">)</span>
    
    <span class="c1"># plot gradient results
</span>    <span class="k">if</span> <span class="n">res</span><span class="p">:</span>

        <span class="c1"># plot [0,0] on bottom
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">alpha</span><span class="o">=</span><span class="mf">0.4</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">black</span><span class="sh">'</span><span class="p">,</span><span class="n">edgecolors</span><span class="o">=</span><span class="sh">'</span><span class="s">white</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">(0,0)</span><span class="sh">'</span><span class="p">)</span>

        <span class="c1"># plot [shift matrix] by gradient descending on bottom
</span>        <span class="n">_x</span><span class="p">,</span><span class="n">_y</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">res</span><span class="p">)[:,</span><span class="mi">0</span><span class="p">],</span><span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">res</span><span class="p">)[:,</span><span class="mi">1</span><span class="p">]</span>
        <span class="n">_nmad</span><span class="p">,</span><span class="n">_rmse</span><span class="p">,</span><span class="n">_std</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">res</span><span class="p">)[:,</span><span class="mi">3</span><span class="p">],</span><span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">res</span><span class="p">)[:,</span><span class="mi">4</span><span class="p">],</span><span class="n">np</span><span class="p">.</span><span class="nf">array</span><span class="p">(</span><span class="n">res</span><span class="p">)[:,</span><span class="mi">5</span><span class="p">]</span>
        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">_x</span><span class="p">,</span> <span class="n">_y</span><span class="p">,</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">alpha</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">s</span><span class="o">=</span><span class="mi">7</span><span class="p">,</span><span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">green</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">GDC</span><span class="sh">'</span><span class="p">)</span>

        <span class="c1"># plot shift matrix on z axis
</span>        <span class="k">if</span> <span class="n">zlabel</span> <span class="o">==</span> <span class="sh">'</span><span class="s">NMAD [m]</span><span class="sh">'</span><span class="p">:</span>
            <span class="n">_z</span> <span class="o">=</span> <span class="n">_nmad</span>
        <span class="k">if</span> <span class="n">zlabel</span> <span class="o">==</span> <span class="sh">'</span><span class="s">RMSE [m]</span><span class="sh">'</span><span class="p">:</span>
            <span class="n">_z</span> <span class="o">=</span> <span class="n">_rmse</span>
        <span class="k">if</span> <span class="n">zlabel</span> <span class="o">==</span> <span class="sh">'</span><span class="s">STD [m]</span><span class="sh">'</span><span class="p">:</span>
            <span class="n">_z</span> <span class="o">=</span> <span class="n">_std</span>
        
        <span class="c1">#ax.scatter([xlim[1]]*len(_x), [ylim[1]]*len(_y),_z, marker="x", color='green')
</span>    
    <span class="c1"># plot nuthkaab results
</span>    <span class="k">if</span> <span class="n">res_1</span><span class="p">:</span>
        <span class="c1"># plot [shift matrix] by gradient descending on bottom
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">res_1</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">res_1</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">alpha</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">s</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span><span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">red</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">NuthKaab</span><span class="sh">'</span><span class="p">)</span>

    <span class="c1"># plot reference results
</span>    <span class="k">if</span> <span class="n">res_2</span><span class="p">:</span>
        <span class="c1"># plot [shift matrix] by gradient descending on bottom
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">res_2</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">res_2</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">alpha</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">s</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span><span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">purple</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">GDC_qc</span><span class="sh">'</span><span class="p">)</span>

    <span class="k">if</span> <span class="n">res_3</span><span class="p">:</span>
        <span class="c1"># plot [shift matrix] by gradient descending on bottom
</span>        <span class="n">ax</span><span class="p">.</span><span class="nf">scatter</span><span class="p">(</span><span class="n">res_2</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">res_2</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">zlim</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">alpha</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">s</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span><span class="n">color</span><span class="o">=</span><span class="sh">'</span><span class="s">black</span><span class="sh">'</span><span class="p">,</span><span class="n">label</span><span class="o">=</span><span class="sh">'</span><span class="s">others</span><span class="sh">'</span><span class="p">)</span>

    <span class="n">ax</span><span class="p">.</span><span class="nf">set</span><span class="p">(</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span> <span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span> <span class="n">zlim</span><span class="o">=</span><span class="n">zlim</span><span class="p">,</span>
        <span class="n">xlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">Shifted E [px]</span><span class="sh">'</span><span class="p">,</span> <span class="n">ylabel</span><span class="o">=</span><span class="sh">'</span><span class="s">Shifted N [px]</span><span class="sh">'</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="n">zlabel</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="n">title</span><span class="p">)</span>

    <span class="n">ax</span><span class="p">.</span><span class="nf">legend</span><span class="p">()</span>

</code></pre></div></div> <h2 id="case-1---shifted-dem">Case 1 - Shifted DEM</h2> <p>Nuth &amp; Kaab (2011) showed elevation difference in a 2D scheme with $\alpha$ as the magnitude of the horizontal shift, b as the direction of the shift, which is extremely useful for coregistration when there are enough points and the size and resolution of DEM are relatively manageable.</p> <p><img src="https://i.imgur.com/UzkOHot.png" alt=""/></p> <p>We can also plot the variation of elevation difference in a 3D scheme. In this case, I shifted a DTM10 (1.2, -0.7) px horizontaly and 0.16 m vertically on purpose and co-register it back in a brute test.</p> <ul> <li>Brute test calculated the elevation difference in 0.1 px interval over a window (-2 px, 2 px) on east and north.</li> <li>All coregs are shifted base on NMAD.</li> <li>The points are subsampled to 10000, 3000, 1000.</li> </ul> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">xsnow.misc</span> <span class="kn">import</span> <span class="n">dem_shift</span>

<span class="c1"># Shift a DTM10 (1.2,-0.7) px and 0.16 m on purpose
</span><span class="n">dtm_10_shifted</span> <span class="o">=</span> <span class="nf">dem_shift</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="mf">1.2</span><span class="p">,</span><span class="o">-</span><span class="mf">0.7</span><span class="p">,</span><span class="mf">0.16</span><span class="p">)</span>
<span class="n">dem_tba</span> <span class="o">=</span> <span class="n">dtm_10_shifted</span>
<span class="n">df_ref</span> <span class="o">=</span> <span class="nf">df_sampling_from_dem</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">dem_tba</span><span class="p">,</span><span class="n">samples</span><span class="o">=</span><span class="mi">20000</span><span class="p">)</span>
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">bond</span> <span class="o">=</span> <span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mf">0.1</span><span class="p">)</span>
<span class="n">xlim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ylim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>

<span class="k">for</span> <span class="n">nmad</span><span class="p">,</span> <span class="n">rmse</span><span class="p">,</span> <span class="n">std</span><span class="p">,</span> <span class="n">median</span><span class="p">,</span><span class="n">res</span><span class="p">,</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_qc</span><span class="p">,</span><span class="n">sampling</span> <span class="ow">in</span> <span class="nf">yield_surface_noise</span><span class="p">(</span><span class="n">dem_tba</span><span class="p">,</span><span class="n">df_ref</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">downsampling_list</span><span class="o">=</span><span class="p">[</span><span class="mi">10000</span><span class="p">,</span><span class="mi">3000</span><span class="p">,</span><span class="mi">1000</span><span class="p">],</span><span class="n">gdc_qc</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">z</span><span class="sh">'</span><span class="p">):</span>
    <span class="n">z_lim_nmad</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">*</span><span class="mi">20</span><span class="p">)</span>
    <span class="n">z_lim_rmse</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">-</span><span class="mf">1.5</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">*</span><span class="mi">20</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
    <span class="n">z_lim_std</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">-</span><span class="mf">1.5</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">*</span><span class="mi">20</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>

    <span class="n">fig</span><span class="p">,(</span><span class="n">ax1</span><span class="p">,</span><span class="n">ax2</span><span class="p">,</span><span class="n">ax3</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="p">.</span><span class="nf">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span><span class="n">subplot_kw</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">projection</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">3d</span><span class="sh">"</span><span class="p">})</span>
    <span class="n">fig</span><span class="p">.</span><span class="nf">suptitle</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">shifted pixels of DEM longyearbyen on </span><span class="si">{</span><span class="n">sampling</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">nmad</span><span class="p">,</span> <span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax1</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD</span><span class="sh">'</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span> <span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_nmad</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">rmse</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax2</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_rmse</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">std</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax3</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">STD</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">STD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_std</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="n">plt</span><span class="p">.</span><span class="nf">show</span><span class="p">()</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 10000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.2031,0.7031,-0.1630),0.1780
Running on downsampling. The length of the gdf: 10000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 10000: (-1.2059757056614222, 0.6973771900864857, -0.16427612, 0.17899064941406248)
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_1-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_1-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_1-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_1.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 3000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.2031,0.7031,-0.1630),0.1780
Running on downsampling. The length of the gdf: 3000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 3000: (-1.199313462032361, 0.6898301972399311, -0.16362, 0.1797598205566406)
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_3-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_3-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_3-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_3.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 1000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.1815,0.6815,-0.1622),0.1794
Running on downsampling. The length of the gdf: 1000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_5-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_5-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_5-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_7_5.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <h2 id="case-2---dem-longyearbyen-1990-and-longyearbyen-2009">Case 2 - DEM Longyearbyen 1990 and Longyearbyen 2009</h2> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">xsnow.misc</span> <span class="kn">import</span> <span class="n">df_sampling_from_dem</span>
<span class="kn">from</span> <span class="n">xdem</span> <span class="kn">import</span> <span class="n">examples</span>
<span class="kn">from</span> <span class="n">geoutils</span> <span class="kn">import</span> <span class="n">Vector</span><span class="p">,</span><span class="n">Raster</span>

<span class="n">dem_ref</span> <span class="o">=</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="n">examples</span><span class="p">.</span><span class="nf">get_path</span><span class="p">(</span><span class="sh">"</span><span class="s">longyearbyen_ref_dem</span><span class="sh">"</span><span class="p">))</span>
<span class="n">dem_tba</span> <span class="o">=</span> <span class="n">xdem</span><span class="p">.</span><span class="nc">DEM</span><span class="p">(</span><span class="n">examples</span><span class="p">.</span><span class="nf">get_path</span><span class="p">(</span><span class="sh">"</span><span class="s">longyearbyen_tba_dem</span><span class="sh">"</span><span class="p">))</span>
<span class="n">glacier_mask</span> <span class="o">=</span> <span class="nc">Vector</span><span class="p">(</span><span class="n">examples</span><span class="p">.</span><span class="nf">get_path</span><span class="p">(</span><span class="sh">"</span><span class="s">longyearbyen_glacier_outlines</span><span class="sh">"</span><span class="p">))</span>
<span class="n">inlier_mask</span> <span class="o">=</span> <span class="o">~</span><span class="n">glacier_mask</span><span class="p">.</span><span class="nf">create_mask</span><span class="p">(</span><span class="n">dem_ref</span><span class="p">)</span>

<span class="n">df_ref</span> <span class="o">=</span> <span class="nf">df_sampling_from_dem</span><span class="p">(</span><span class="n">dem_ref</span><span class="p">,</span><span class="n">dem_tba</span><span class="p">,</span><span class="n">inlier_mask</span><span class="o">=</span><span class="n">inlier_mask</span><span class="p">,</span><span class="n">samples</span><span class="o">=</span><span class="mi">15000</span><span class="p">,</span> <span class="n">offset</span><span class="o">=</span><span class="sh">'</span><span class="s">ul</span><span class="sh">'</span><span class="p">)</span>
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">bond</span> <span class="o">=</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mf">0.05</span><span class="p">)</span>

<span class="c1"># axes limit
</span><span class="n">xlim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">)</span>
<span class="n">ylim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">)</span>

<span class="k">for</span> <span class="n">nmad</span><span class="p">,</span> <span class="n">rmse</span><span class="p">,</span> <span class="n">std</span><span class="p">,</span> <span class="n">median</span><span class="p">,</span><span class="n">res</span><span class="p">,</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_qc</span><span class="p">,</span><span class="n">sampling</span> <span class="ow">in</span> <span class="nf">yield_surface_noise</span><span class="p">(</span><span class="n">dem_tba</span><span class="p">,</span> <span class="n">df_ref</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">downsampling_list</span><span class="o">=</span><span class="p">[</span><span class="mi">10000</span><span class="p">,</span><span class="mi">3000</span><span class="p">,</span><span class="mi">1000</span><span class="p">],</span><span class="n">gdc_qc</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="n">z_name</span><span class="o">=</span><span class="sh">'</span><span class="s">z</span><span class="sh">'</span><span class="p">):</span>
    <span class="n">z_lim_nmad</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">*</span><span class="mi">3</span><span class="p">)</span>
    <span class="n">z_lim_rmse</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">-</span><span class="mf">1.5</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">*</span><span class="mi">3</span><span class="p">)</span>
    <span class="n">z_lim_std</span> <span class="o">=</span> <span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">-</span><span class="mf">1.5</span><span class="p">),</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">*</span><span class="mi">3</span><span class="p">)</span>

    <span class="n">fig</span><span class="p">,(</span><span class="n">ax1</span><span class="p">,</span><span class="n">ax2</span><span class="p">,</span><span class="n">ax3</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="p">.</span><span class="nf">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span><span class="n">subplot_kw</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">projection</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">3d</span><span class="sh">"</span><span class="p">})</span>
    <span class="n">fig</span><span class="p">.</span><span class="nf">suptitle</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">shifted pixels of DEM longyearbyen on </span><span class="si">{</span><span class="n">sampling</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">nmad</span><span class="p">,</span> <span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax1</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD</span><span class="sh">'</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span> <span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_nmad</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">rmse</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax2</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_rmse</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">std</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax3</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">STD</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">STD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_std</span><span class="p">,</span><span class="n">zlim_s</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
    <span class="n">plt</span><span class="p">.</span><span class="nf">show</span><span class="p">()</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 10000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.5469,-0.1250,-1.8884),2.4996
Running on downsampling. The length of the gdf: 10000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 10000: (-0.44989970346639757, -0.13631736964576785, -1.9579468, 2.543117335510254)
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_1-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_1-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_1-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_1.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 3000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.5000,-0.1562,-1.8764),2.5242
Running on downsampling. The length of the gdf: 3000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 3000: (-0.43152742135357697, -0.17112353846215173, -1.9326782, 2.547257286071777)
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_3-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_3-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_3-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_3.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 1000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.5000,-0.1250,-1.8980),2.5237
Running on downsampling. The length of the gdf: 1000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_5-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_5-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_5-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_10_5.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <h2 id="case-3---dtm10">Case 3 - DTM10</h2> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">bond</span> <span class="o">=</span> <span class="p">(</span><span class="o">-</span><span class="mf">1.0</span><span class="p">,</span><span class="mf">1.0</span><span class="p">,</span><span class="mf">0.05</span><span class="p">)</span>

<span class="c1"># axes limit
</span><span class="n">xlim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mf">1.0</span><span class="p">,</span><span class="mf">0.5</span><span class="p">)</span>
<span class="n">ylim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mf">0.5</span><span class="p">,</span><span class="mf">1.0</span><span class="p">)</span>

<span class="k">for</span> <span class="n">nmad</span><span class="p">,</span> <span class="n">rmse</span><span class="p">,</span> <span class="n">std</span><span class="p">,</span> <span class="n">median</span><span class="p">,</span><span class="n">res</span><span class="p">,</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_qc</span><span class="p">,</span><span class="n">sampling</span> <span class="ow">in</span> <span class="nf">yield_surface_noise</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_subset_dtm10</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">downsampling_list</span><span class="o">=</span><span class="p">[</span><span class="mi">10000</span><span class="p">,</span><span class="mi">3000</span><span class="p">,</span><span class="mi">1000</span><span class="p">],</span><span class="n">weight</span><span class="o">=</span><span class="sh">'</span><span class="s">subset_te_flag</span><span class="sh">'</span><span class="p">):</span>
    <span class="n">z_lim_nmad</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="p">)</span>
    <span class="n">z_lim_rmse</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
    <span class="n">z_lim_std</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>  

    <span class="n">fig</span><span class="p">,(</span><span class="n">ax1</span><span class="p">,</span><span class="n">ax2</span><span class="p">,</span><span class="n">ax3</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="p">.</span><span class="nf">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span><span class="n">subplot_kw</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">projection</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">3d</span><span class="sh">"</span><span class="p">})</span>
    <span class="n">fig</span><span class="p">.</span><span class="nf">suptitle</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">shifted pixels of DTM10 on </span><span class="si">{</span><span class="n">sampling</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">nmad</span><span class="p">,</span> <span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax1</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD</span><span class="sh">'</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span> <span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_nmad</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">rmse</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax2</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_rmse</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">std</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax3</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">STD</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">STD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_std</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
    <span class="n">plt</span><span class="p">.</span><span class="nf">show</span><span class="p">()</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 10000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4375,0.2656,-0.0065),0.5992
Running on downsampling. The length of the gdf: 10000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 10000: (-0.43959885000839216, 0.2685718552192047, -0.0128173828125, 0.5952479736328125)
Running best_shift_px on downsampling. The length of the gdf: 10000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4844,0.3125,0.1088),0.3848
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_1-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_1-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_1-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_1.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 3000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4844,0.1815,-0.0063),0.6086
Running on downsampling. The length of the gdf: 3000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
NuthKaab coreg on 3000: (-0.4273086350909474, 0.259520889514669, -0.01318359375, 0.5863798828125)
Running best_shift_px on downsampling. The length of the gdf: 3000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4591,0.2909,0.1092),0.3856
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_3-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_3-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_3-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_3.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 1000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4688,0.5060,-0.0110),0.6305
Running on downsampling. The length of the gdf: 1000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
Running best_shift_px on downsampling. The length of the gdf: 1000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-0.4627,0.3438,0.1067),0.3884
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_5-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_5-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_5-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_13_5.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">xsnow.godh</span> <span class="kn">import</span> <span class="n">get_dh_by_shift_px_gdf</span>

<span class="nf">get_dh_by_shift_px_gdf</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_subset_dtm10</span><span class="p">,(</span><span class="o">-</span><span class="mf">0.4375</span><span class="p">,</span><span class="mf">0.2656</span><span class="p">),</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0.5992295654296874
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">get_dh_by_shift_px_gdf</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_subset_dtm10</span><span class="p">,(</span><span class="o">-</span><span class="mf">0.43959885000839216</span><span class="p">,</span> <span class="mf">0.2685718552192047</span><span class="p">),</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0.5986866210937499
</code></pre></div></div> <h2 id="case---4-dtm1">Case - 4 DTM1</h2> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">bond</span> <span class="o">=</span> <span class="p">(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mf">0.1</span><span class="p">)</span>

<span class="c1"># axes limit
</span><span class="n">xlim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">)</span>
<span class="n">ylim</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">)</span>

<span class="k">for</span> <span class="n">nmad</span><span class="p">,</span> <span class="n">rmse</span><span class="p">,</span> <span class="n">std</span><span class="p">,</span> <span class="n">median</span><span class="p">,</span><span class="n">res</span><span class="p">,</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_qc</span><span class="p">,</span><span class="n">sampling</span> <span class="ow">in</span> <span class="nf">yield_surface_noise</span><span class="p">(</span><span class="n">dtm_1</span><span class="p">,</span><span class="n">sf_subset_dtm1</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">downsampling_list</span><span class="o">=</span><span class="p">[</span><span class="mi">10000</span><span class="p">,</span><span class="mi">1000</span><span class="p">,</span><span class="mi">500</span><span class="p">],</span><span class="n">gdc_qc</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="n">weight</span><span class="o">=</span><span class="sh">'</span><span class="s">subset_te_flag</span><span class="sh">'</span><span class="p">):</span>
    <span class="n">z_lim_nmad</span> <span class="o">=</span> <span class="p">(</span><span class="mf">0.4</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">3</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="o">-</span><span class="mf">0.4</span><span class="p">)</span>
    <span class="n">z_lim_rmse</span> <span class="o">=</span> <span class="p">(</span><span class="mf">0.6</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">4</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="o">-</span><span class="mf">0.8</span><span class="p">)</span>
    <span class="n">z_lim_std</span> <span class="o">=</span> <span class="p">(</span><span class="mf">0.6</span><span class="p">,</span><span class="n">res</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span><span class="o">-</span><span class="mf">0.8</span><span class="p">)</span>

    <span class="n">fig</span><span class="p">,(</span><span class="n">ax1</span><span class="p">,</span><span class="n">ax2</span><span class="p">,</span><span class="n">ax3</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="p">.</span><span class="nf">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span><span class="n">subplot_kw</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">projection</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">3d</span><span class="sh">"</span><span class="p">})</span>
    <span class="n">fig</span><span class="p">.</span><span class="nf">suptitle</span><span class="p">(</span><span class="sa">f</span><span class="sh">'</span><span class="s">shifted pixels of DTM1 on </span><span class="si">{</span><span class="n">sampling</span><span class="si">}</span><span class="s"> points</span><span class="sh">'</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">nmad</span><span class="p">,</span> <span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax1</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD</span><span class="sh">'</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span> <span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">NMAD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_nmad</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">rmse</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax2</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">RMSE [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_rmse</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span>
    <span class="nf">surface_plot</span><span class="p">(</span><span class="n">std</span><span class="p">,</span><span class="n">bond</span><span class="o">=</span><span class="n">bond</span><span class="p">,</span><span class="n">xlim</span><span class="o">=</span><span class="n">xlim</span><span class="p">,</span><span class="n">ylim</span><span class="o">=</span><span class="n">ylim</span><span class="p">,</span><span class="n">ax</span><span class="o">=</span><span class="n">ax3</span><span class="p">,</span><span class="n">title</span><span class="o">=</span><span class="sh">'</span><span class="s">STD</span><span class="sh">'</span><span class="p">,</span><span class="n">res</span><span class="o">=</span><span class="n">res</span><span class="p">,</span><span class="n">res_1</span><span class="o">=</span><span class="n">res_nk</span><span class="p">,</span><span class="n">res_2</span><span class="o">=</span><span class="n">res_qc</span><span class="p">,</span> <span class="n">zlabel</span><span class="o">=</span><span class="sh">'</span><span class="s">STD [m]</span><span class="sh">'</span><span class="p">,</span><span class="n">zlim</span><span class="o">=</span><span class="n">z_lim_std</span><span class="p">,</span><span class="n">z_step_level</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span>
    <span class="n">plt</span><span class="p">.</span><span class="nf">show</span><span class="p">()</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.5000,-1.9375,-0.0508),0.5833
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_1-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_1-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_1-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_1.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 1000
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.7656,0.2284,-0.0659),0.5798
Running on downsampling. The length of the gdf: 1000
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_3-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_3-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_3-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_3.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Running best_shift_px on downsampling. The length of the gdf: 500
Gradient Descending Coreg fit matrix(e_px,n_px,bias),nmad:(-1.9736,-2.0781,-0.0555),0.5788
Running on downsampling. The length of the gdf: 500
Set downsampling = other value or False to make a change.
Warning: There is no curvature in dataframe. Set mask_highcurv = True for more robust results and costs more time
</code></pre></div></div> <div class="row"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_5-480.webp 480w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_5-800.webp 800w,/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_5-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/notebook/xsnow_coreg/xsnow_coreg_18_5.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" data-zoomable="" loading="lazy" onerror="this.onerror=null; document.querySelectorAll('.responsive-img-srcset').forEach(function (n) { n.remove(); });"/> </picture> </figure> </div> </div> <h2 id="the-best-fit-algorithm-and-coefficients">The Best Fit Algorithm and Coefficients</h2> <p>Gradient descending can also be implemented by L-BFGS-B from Scipy.optimize if the noise is not severe.</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">scipy.optimize</span> <span class="kn">import</span> <span class="n">minimize</span>
<span class="kn">from</span> <span class="n">xsnow.godh</span> <span class="kn">import</span> <span class="n">get_dh_by_shift_px_gdf</span>

<span class="c1"># start iteration, find the best shifting px
</span><span class="n">func_x</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nf">get_dh_by_shift_px_gdf</span><span class="p">(</span><span class="n">dtm_10</span><span class="p">,</span><span class="n">sf_subset_dtm10</span><span class="p">,</span><span class="n">x</span><span class="p">,</span><span class="n">perc_t</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span><span class="n">std_t</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span><span class="n">stat</span><span class="o">=</span><span class="sh">'</span><span class="s">rmse</span><span class="sh">'</span><span class="p">)</span>
<span class="n">res</span> <span class="o">=</span> <span class="nf">minimize</span><span class="p">(</span><span class="n">func_x</span><span class="p">,</span> <span class="n">x0</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">),</span> <span class="n">method</span><span class="o">=</span><span class="sh">'</span><span class="s">L-BFGS-B</span><span class="sh">'</span><span class="p">,</span><span class="n">bounds</span><span class="o">=</span><span class="p">((</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">)))</span>
<span class="n">res</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>      fun: 1.5511970187060895
 hess_inv: &lt;2x2 LbfgsInvHessProduct with dtype=float64&gt;
      jac: array([0.        , 0.01073706])
  message: 'CONVERGENCE: REL_REDUCTION_OF_F_&lt;=_FACTR*EPSMCH'
     nfev: 99
      nit: 7
     njev: 33
   status: 0
  success: True
        x: array([-0.09557894,  0.00446687])
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">res</span> <span class="o">=</span> <span class="nf">minimize</span><span class="p">(</span><span class="n">func_x</span><span class="p">,</span> <span class="n">x0</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">),</span> <span class="n">method</span><span class="o">=</span><span class="sh">'</span><span class="s">BFGS</span><span class="sh">'</span><span class="p">)</span>
<span class="n">res</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>      fun: 1.5797875171900577
 hess_inv: array([[7.02123e-09, 0.00000e+00],
       [0.00000e+00, 1.00000e+00]])
      jac: array([0.0354276, 0.       ])
  message: 'Desired error not necessarily achieved due to precision loss.'
     nfev: 237
      nit: 2
     njev: 75
   status: 2
  success: False
        x: array([1.37243807e-07, 0.00000000e+00])
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="n">xsnow.noiseopt</span> <span class="kn">import</span> <span class="n">minimizeCompass</span>
<span class="n">res</span> <span class="o">=</span> <span class="nf">minimizeCompass</span><span class="p">(</span><span class="n">func_x</span><span class="p">,</span> <span class="n">x0</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">),</span> <span class="n">deltainit</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span><span class="n">deltatol</span><span class="o">=</span><span class="mf">0.006</span><span class="p">,</span><span class="n">feps</span><span class="o">=</span><span class="mf">0.0001</span><span class="p">,</span><span class="n">bounds</span><span class="o">=</span><span class="p">((</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">)),</span><span class="n">errorcontrol</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">res</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    free: array([False, False])
     fun: 1.491898243075958
 message: 'convergence within deltatol'
    nfev: 41
     nit: 14
 success: True
       x: array([-0.359375,  0.25    ])
</code></pre></div></div> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">res</span> <span class="o">=</span> <span class="nf">minimizeCompass</span><span class="p">(</span><span class="n">func_x</span><span class="p">,</span> <span class="n">x0</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">),</span> <span class="n">deltainit</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span><span class="n">deltatol</span><span class="o">=</span><span class="mf">0.006</span><span class="p">,</span><span class="n">feps</span><span class="o">=</span><span class="mf">0.0005</span><span class="p">,</span><span class="n">bounds</span><span class="o">=</span><span class="p">((</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),(</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">)),</span><span class="n">errorcontrol</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">res</span>
</code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    free: array([ True, False])
     fun: 1.491898243075958
 message: 'convergence within deltatol. dim 0 is free at optimum'
    nfev: 40
     nit: 13
 success: True
       x: array([-0.359375,  0.25    ])
</code></pre></div></div>]]></content><author><name>Zhihao</name></author><category term="notes"/><category term="geodata"/><category term="datascience"/><summary type="html"><![CDATA[but suffering from noise]]></summary></entry></feed>