Skip to content

Commit 0e5afe1

Browse files
armanbilgearmanbilge
committed
deploy: 4bf694f 
1 parent c19998b commit 0e5afe1

File tree

2 files changed

+29
-59
lines changed

2 files changed

+29
-59
lines changed

‎blog/typelevel-native.html‎

Lines changed: 29 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ <h4 id="event-loop-runtime" class="section"><a class="anchor-link" href="#event-
132132
<p>Fortunately Daniel Spiewak had the fantastic insight that the “dummy runtime” which I created to initially cross-build Cats Effect for Native could be reformulated into a legitimate event loop implementation by extending it with the capability to “poll” for I/O events: a <code>PollingExecutorScheduler</code>.</p>
133133
<p>The <a href="https://github.com/typelevel/cats-effect/blob/7ca03db50342773a79a01ecf137d953408ac6b1d/core/native/src/main/scala/cats/effect/unsafe/PollingExecutorScheduler.scala"><code>PollingExecutorScheduler</code></a> implements both <a href="https://www.scala-lang.org/api/2.13.8/scala/concurrent/ExecutionContext.html"><code>ExecutionContext</code></a> and <a href="https://github.com/typelevel/cats-effect/blob/236a0db0e95be829de34d7a8e3c06914738b7b06/core/shared/src/main/scala/cats/effect/unsafe/Scheduler.scala"><code>Scheduler</code></a> and maintains two queues:</p>
134134
<ul>
135-
<li>a queue of tasks (read: fibers) to execute </li>
135+
<li>a queue of tasks (read: fibers) to execute</li>
136136
<li>a priority queue of timers (read: <code>IO.sleep(...)</code>), sorted by expiration</li>
137137
</ul>
138138
<p>It also defines an abstract method:</p>
@@ -174,7 +174,7 @@ <h4 id="non-blocking-i-o" class="section"><a class="anchor-link" href="#non-bloc
174174
<p>After creating an epoll instance (identified by a file descriptor) we can register sockets (also identified by file descriptors) with <code>epoll_ctl</code>. Typically we will register to be notified of the “read-ready” (<code>EPOLLIN</code>) and “write-ready” (<code>EPOLLOUT</code>) events on that socket. Finally, the actual polling is implemented with <code>epoll_wait</code>, which sleeps until the next I/O event is ready or the <code>timeout</code> expires. Thus we can use it to implement a <code>PollingExecutorScheduler</code>.</p>
175175
<p>As previously mentioned, these sorts of polling APIs are ubiquitous and not just for working directly with sockets. For example, <a href="https://curl.se/libcurl/">libcurl</a> (the C library behind the well-known CLI) exposes a function for polling for I/O on all ongoing HTTP requests.</p>
176176
<pre><code class="c">#include &lt;curl/curl.h&gt;
177-
177+
178178
CURLMcode curl_multi_poll(CURLM *multi_handle,
179179
struct curl_waitfd extra_fds[],
180180
unsigned int extra_nfds,
@@ -208,77 +208,47 @@ <h4 id="tls" class="section"><a class="anchor-link" href="#tls"><svg xmlns="http
208208
<h3 id="what-s-next-and-how-can-i-get-involved" class="section"><a class="anchor-link" href="#what-s-next-and-how-can-i-get-involved"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc. --><path fill="currentColor" d="M419.5 96c-16.6 0-32.7 4.5-46.8 12.7-15.8-16-34.2-29.4-54.5-39.5 28.2-24 64.1-37.2 101.3-37.2 86.4 0 156.5 70 156.5 156.5 0 41.5-16.5 81.3-45.8 110.6l-71.1 71.1c-29.3 29.3-69.1 45.8-110.6 45.8-86.4 0-156.5-70-156.5-156.5 0-1.5 0-3 .1-4.5 .5-17.7 15.2-31.6 32.9-31.1s31.6 15.2 31.1 32.9c0 .9 0 1.8 0 2.6 0 51.1 41.4 92.5 92.5 92.5 24.5 0 48-9.7 65.4-27.1l71.1-71.1c17.3-17.3 27.1-40.9 27.1-65.4 0-51.1-41.4-92.5-92.5-92.5zM275.2 173.3c-1.9-.8-3.8-1.9-5.5-3.1-12.6-6.5-27-10.2-42.1-10.2-24.5 0-48 9.7-65.4 27.1L91.1 258.2c-17.3 17.3-27.1 40.9-27.1 65.4 0 51.1 41.4 92.5 92.5 92.5 16.5 0 32.6-4.4 46.7-12.6 15.8 16 34.2 29.4 54.6 39.5-28.2 23.9-64 37.2-101.3 37.2-86.4 0-156.5-70-156.5-156.5 0-41.5 16.5-81.3 45.8-110.6l71.1-71.1c29.3-29.3 69.1-45.8 110.6-45.8 86.6 0 156.5 70.6 156.5 156.9 0 1.3 0 2.6 0 3.9-.4 17.7-15.1 31.6-32.8 31.2s-31.6-15.1-31.2-32.8c0-.8 0-1.5 0-2.3 0-33.7-18-63.3-44.8-79.6z"/></svg></a>What’s next and how can I get involved?</h3>
209209
<p>Please try the Typelevel Native stack! And even better deploy it, and do so loudly!</p>
210210
<p>Besides that, here is a brain-dump of project ideas and existing projects that would love contributors. I am happy to help folks get started on any of these, or ideas of your own!</p>
211-
<ul>
212-
<li>Creating example applications, templates, and tutorials:</li>
213-
</ul>
214-
<ul>
215-
<li>If you are short on inspiration, try cross-building existing examples such as <a href="https://github.com/typelevel/fs2-chat/">fs2-chat</a>, <a href="https://github.com/djspiewak/kitteh-redis">kitteh-redis</a>, <a href="https://github.com/keynmol/jobby/">Jobby</a>.</li>
216-
</ul>
217211
<ul>
218212
<li>
219-
<p>Spread the word: <a href="https://youforgotapercentagesignoracolon.com/">you-forgot-a-percentage-sign-or-a-colon</a>.</p>
213+
<p>Creating example applications, templates, and tutorials:</p>
214+
<ul>
215+
<li>If you are short on inspiration, try cross-building existing examples such as <a href="https://github.com/typelevel/fs2-chat/">fs2-chat</a>, <a href="https://github.com/djspiewak/kitteh-redis">kitteh-redis</a>, <a href="https://github.com/keynmol/jobby/">Jobby</a>.</li>
216+
<li>Spread the word: <a href="https://youforgotapercentagesignoracolon.com/">you-forgot-a-percentage-sign-or-a-colon</a>.</li>
217+
</ul>
220218
</li>
221219
<li>
222220
<p>Cross-building existing libraries and developing new, Typelevel-stack ones:</p>
223-
</li>
224-
</ul>
225-
<ul>
226-
<li>Go <a href="https://github.com/typelevel/feral">feral</a> and implement a pure Scala <a href="https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html">custom AWS Lambda runtime</a> that cross-builds for Native.</li>
227-
</ul>
228-
<ul>
229-
<li>A pure Scala <a href="https://grpc.io/">gRPC</a> implementation built on http4s would be fantastic, even for the JVM. Christopher Davenport has published a <a href="https://github.com/ChristopherDavenport/grpc-playground">proof-of-concept</a>.</li>
230-
</ul>
231-
<ul>
232-
<li><a href="https://github.com/satabin/fs2-data/">fs2-data</a> has pure Scala support for a plethora of data formats. The <a href="https://github.com/http4s/http4s-fs2-data">http4s-fs2-data</a> integration needs your help to get off the ground!</li>
233-
</ul>
234-
<ul>
235-
<li>
236-
<p>Lack of cross-platform cryptography is one of the remaining sore points in cross-building. I started the <a href="https://github.com/typelevel/bobcats">bobcats</a> project to fill the gap but I am afraid it needs love from a more dedicated maintainer.</p>
221+
<ul>
222+
<li>Go <a href="https://github.com/typelevel/feral">feral</a> and implement a pure Scala <a href="https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html">custom AWS Lambda runtime</a> that cross-builds for Native.</li>
223+
<li>A pure Scala <a href="https://grpc.io/">gRPC</a> implementation built on http4s would be fantastic, even for the JVM. Christopher Davenport has published a <a href="https://github.com/ChristopherDavenport/grpc-playground">proof-of-concept</a>.</li>
224+
<li><a href="https://github.com/satabin/fs2-data/">fs2-data</a> has pure Scala support for a plethora of data formats. The <a href="https://github.com/http4s/http4s-fs2-data">http4s-fs2-data</a> integration needs your help to get off the ground!</li>
225+
<li>Lack of cross-platform cryptography is one of the remaining sore points in cross-building. I started the <a href="https://github.com/typelevel/bobcats">bobcats</a> project to fill the gap but I am afraid it needs love from a more dedicated maintainer.</li>
226+
</ul>
237227
</li>
238228
<li>
239229
<p>Integrations with native libraries:</p>
240-
</li>
241-
</ul>
242-
<ul>
243-
<li>I kick-started <a href="https://github.com/http4s/http4s-curl/">http4s-curl</a> and would love to see someone take the reigns!</li>
244-
</ul>
245-
<ul>
246-
<li>An <a href="https://unit.nginx.org/">NGINX Unit</a> server backend for http4s promises exceptional performance. <a href="https://github.com/lolgab/snunit">snunit</a> pioneered this approach.</li>
247-
</ul>
248-
<ul>
249-
<li>Using <a href="https://github.com/cloudflare/quiche">quiche</a> for HTTP/3 looks yummy!</li>
250-
</ul>
251-
<ul>
252-
<li>
253-
<p>An idiomatic wrapper for <a href="https://www.sqlite.org/index.html">SQLite</a>. See also <a href="https://github.com/davenverse/sqlite-sjs/pull/1">davenverse/sqlite-sjs#1</a> which proposes cross-platform API backed by Doobie on the JVM.</p>
230+
<ul>
231+
<li>I kick-started <a href="https://github.com/http4s/http4s-curl/">http4s-curl</a> and would love to see someone take the reigns!</li>
232+
<li>An <a href="https://unit.nginx.org/">NGINX Unit</a> server backend for http4s promises exceptional performance. <a href="https://github.com/lolgab/snunit">snunit</a> pioneered this approach.</li>
233+
<li>Using <a href="https://github.com/cloudflare/quiche">quiche</a> for HTTP/3 looks yummy!</li>
234+
<li>An idiomatic wrapper for <a href="https://www.sqlite.org/index.html">SQLite</a>. See also <a href="https://github.com/davenverse/sqlite-sjs/pull/1">davenverse/sqlite-sjs#1</a> which proposes cross-platform API backed by Doobie on the JVM.</li>
235+
</ul>
254236
</li>
255237
<li>
256238
<p>Developing I/O-integrated runtimes:</p>
257-
</li>
258-
</ul>
259-
<ul>
260-
<li><a href="https://github.com/armanbilge/epollcat">epollcat</a> supports Linux and macOS and has plenty of opportunity for optimization and development.</li>
261-
</ul>
262-
<ul>
263-
<li>A <a href="https://github.com/libuv/libuv/">libuv</a>-based runtime would have solid cross-OS support, including Windows. Prior art in <a href="https://github.com/scala-native/scala-native-loop/">scala-native-loop</a>.</li>
264-
</ul>
265-
<ul>
266-
<li>
267-
<p>Personally I am excited to work on an <a href="https://en.wikipedia.org/wiki/Io_uring">io_uring</a> runtime.</p>
239+
<ul>
240+
<li><a href="https://github.com/armanbilge/epollcat">epollcat</a> supports Linux and macOS and has plenty of opportunity for optimization and development.</li>
241+
<li>A <a href="https://github.com/libuv/libuv/">libuv</a>-based runtime would have solid cross-OS support, including Windows. Prior art in <a href="https://github.com/scala-native/scala-native-loop/">scala-native-loop</a>.</li>
242+
<li>Personally I am excited to work on an <a href="https://en.wikipedia.org/wiki/Io_uring">io_uring</a> runtime.</li>
243+
</ul>
268244
</li>
269245
<li>
270246
<p>Tooling. Anton Sviridov has spear-headed two major projects in this area:</p>
271-
</li>
272-
</ul>
273-
<ul>
274-
<li><a href="https://github.com/indoorvivants/sbt-vcpkg/">sbt-vcpkg</a> is working hard to solve the native dependency problem.</li>
275-
</ul>
276-
<ul>
277-
<li><a href="https://github.com/indoorvivants/sn-bindgen">sn-bindgen</a> generates Scala Native bindings to native libraries directly from <code>*.h</code> header files. I found it immensely useful while working on http4s-curl, epollcat, and the s2n-tls integration in FS2.</li>
278-
</ul>
279-
<ul>
280-
<li>
281-
<p>Also: we are <em>badly</em> in need of a pure Scala port of the <a href="https://github.com/openjdk/jmh">Java Microbenchmark Harness</a>. Not the whole thing obviously, but just enough to run the existing Cats Effect benchmarks for example.</p>
247+
<ul>
248+
<li><a href="https://github.com/indoorvivants/sbt-vcpkg/">sbt-vcpkg</a> is working hard to solve the native dependency problem.</li>
249+
<li><a href="https://github.com/indoorvivants/sn-bindgen">sn-bindgen</a> generates Scala Native bindings to native libraries directly from <code>*.h</code> header files. I found it immensely useful while working on http4s-curl, epollcat, and the s2n-tls integration in FS2.</li>
250+
<li>Also: we are <em>badly</em> in need of a pure Scala port of the <a href="https://github.com/openjdk/jmh">Java Microbenchmark Harness</a>. Not the whole thing obviously, but just enough to run the existing Cats Effect benchmarks for example.</li>
251+
</ul>
282252
</li>
283253
<li>
284254
<p>Scala Native itself. Lots to do there!</p>

‎search/searchIndex.idx‎

-13 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)