总结

以上是小编为你收集整理的Apache CGIredirect到绝对URI不起作用全部内容。

如果觉得小编网站内容还不错，欢迎将小编网站推荐给好友。

Distributed PostgreSQL on a Google Spanner Architecture – Query Layer

转自：https://blog.yugabyte.com/distributed-postgresql-on-a-google-spanner-architecture-query-layer/

Our previous post dived into the details of the storage layer of YugaByte DB called DocDB, a distributed document store inspired by Google Spanner. This post focuses on YugaByte SQL (YSQL), a distributed, highly resilient, PostgreSQL-compatible SQL API layer powered by DocDB. A follow-up post will highlight the challenges faced and lessons learned when engineering such a database.

YSQL, Distributed PostgreSQL Made Real

YugaByte SQL (YSQL) is a distributed and highly resilient SQL layer, running across multiple nodes. It is compatible with the SQL dialect and wire protocol of PostgreSQL. This means that developers familiar with PostgreSQL can fully reuse their knowledge (and the standard PostgreSQL client drivers) to build an application powered by YSQL.

YSQL essentially transforms the monolithic PostgreSQL database into a DocDB-powered distributed database. To accomplish this, it reuses open source PostgreSQL’s query layer (written in C) as much as possible.

Following were the design goals we set for YSQL early on.

• Reuse the open source, mature and feature-rich PostgreSQL query layer
• Preserve existing PostgreSQL functionality and extend as necessary
• Enable migrations to newer versions of PostgreSQL by implementing features in a modular approach

Relentless execution towards the above goals has paid rich dividends. YSQL now supports a wider range of existing PostgreSQL functionality than we had originally expected. This is evident from the v1.2 feature matrix, examples being:

• DDL statements: CREATE, DROP and TRUNCATE tables
• Data types: All primitive types including numeric types (integers and floats), text data types, byte arrays, date-time types, UUID, SERIAL, as well as JSONB
• DML statements: Most statements such as INSERT, UPDATE, SELECT and DELETE. Bulk of core SQL functionality now supported includes JOINs, WHERE clauses, GROUP BY, ORDER BY, LIMIT, OFFSETand SEQUENCES
• Transactions: ABORT, ROLLBACK, BEGIN, END, and COMMIT
• Expressions: Rich set of PostgreSQL built-in functions and operators
• Other Features: VIEWs, EXPLAIN, PREPARE-BIND-EXECUTE, and JDBC support

As for the design goal of migrating to newer versions, YSQL started with the PostgreSQL v10.4 and recently rebased to PostgreSQL v11.2 in a matter of weeks!

How YSQL Works?

YSQL internals can be categorized into four distinct areas:

• System catalog management
• User table management
• The read and write IO Path
• Mapping SQL tables to a document store

The next sections detail each of the above areas. Before diving into the details, here’s a quick recap of DocDB from the first post of this series.

• Every table in DocDB has the same schema: one key maps to one document.
• As a distributed database, it replicates data on each write.
• Offers single-key linearizability and multi-key snapshot isolation (serializable isolation is in the works).
• Native support for secondary indexes on any document attribute.
• Efficient querying and updating a subset of attributes of any document.

System Catalog Management

The PostgreSQL documentation on system catalogs says that the system catalogs are regular tables where schema metadata is stored, such as information about tables and columns, and internal bookkeeping information. The initdb code path in PostgreSQL, which is completely different from the code path the deals with user tables, creates and initializes system catalog tables. So, in order to make a distributed SQL database with no single points of failure, it is essential to replicate these system catalogs.

1. Initialize system catalog through initdb

When YSQL starts up for the first time, a modified initdb executes and creates the system catalog a replicated, single-tablet system catalog table in DocDB. This is shown in the figure above.
The system catalog tablets in DocDB forms a Raft group, which replicates data onto a set of nodes and can tolerate failures. In the figure above, the system catalog tablet leader is shown with a solid border while the followers are shown with a dotted border. This ensures that PostgreSQL can still rely on the familiar system catalog in order to function.

2. Ready to serve apps

Once the system catalogs are created, YSQL can be used by applications. Since the data is replicated across nodes and persisted on disk, initdb is not needed on subsequent restarts of the cluster.

User Table Management

Now that the YSQL cluster is up and running, let us consider the scenario when a user creates a table. This happens in the following four steps.

1. Parse and analyze the query

Just as with PostgreSQL, the query is received by PostgreSQL server process – which parses, analyzes and executes the query.

2. Route query to tablet leader of DocDB system catalog

In the case of a regular PostgreSQL, the execution phase would add entries to the system catalog tables and create some directories and files on the local filesystem. In the case of YSQL, this update to the system catalog is sent to the tablet leader of the distributed system catalog table in DocDB.

3. Replicate system catalog entry across nodes in DocDB

The tablet leader of the distributed system catalog table in DocDB is responsible replicating the update to the followers. This is done using Raft consensus, which ensures that the update is linearizable even in the presence of faults.

4. Create user table in DocDB

Now that the entry has been persisted in the system catalog, the next step of the execution phase is to create a distributed DocDB table. This involves creating a number of tablets (which have replicas) across a set of nodes. This is shown in the diagram below.

Once the above steps are complete, the table is ready to use.

Read/Write IO Path

The read and write IO paths are quite similar. Let us understand the write IO path, which involves replication of data in DocDB. The read IO path is similar, except for the last step which can serve data directly from the leader of the tablet in DocDB.

1. Parse and analyze the query

Just as with PostgreSQL, the PostgreSQL server process receives the query. It then goes through the parser, analyzer, planner and the executor. Some of the planning, analysis and execution steps, however, are different to accommodate a distributed database instead of the local store.

2. Route the insert to the tablet leader

The SQL insert statement may end up updating a single row or multiple rows. Although DocDB can handle both cases natively, these two cases are detected and handled differently to improve the performance of YSQL. Single row inserts are routed directly to the tablet leader that owns the primary key of that row. Inserts affecting multiple rows are sent to a global transaction manager which performs a distributed transaction. The single-row insert case is shown below.

3. Replicate the write through Raft

In the of single-row inserts, the tablet leader replicates the data using the Raft protocol onto the followers. This simpler case is shown below. In the case of multi-row inserts, the global transaction manager writes multiple records (transaction status records, provisional records, etc) across tablets (often on different nodes). Each of these writes are replicated using Raft consensus. The hybrid logical clock or HLC tracking in the cluster serves as a coarsely synchronized, highly available global clock to coordinate writes. This results in the writes being fault tolerant, with a high-performance system.

Mapping SQL Tables to Documents

Each user table in YSQL maps to a corresponding DocDB table with multiple tablets. The YSQL tables come with their own schemas, while all the DocDB tables have the same schema, which is shown below. The actual schema enforcement is done using table schema metadata.

The combined set of primary key column values are used to construct the DocKey above. Each of the value columns (non-primary key columns) are mapped to one attribute in the Document Value above.

The various YSQL constructs are mapped to suitable DocDB equivalents. This is shown in the table below.

So how does this look in practice? Let us take an example. Consider the following rather simple table.

This will correspond to a DocDB table that has a document key to value schema. Now, lets us perform the following insert at time T1.

This will get translated into the following entries in the DocDB table.

YSQL Benefits

A YSQL cluster appears as a single logical PostgreSQL database to applications. All nodes in the YSQL layer are identical and application clients can connect to any node in order to read or write data. Along with maximum PostgreSQL compatibility, such an architecture delivers a number of benefits.

Horizontal Write Scalability

Since DocDB is capable of being scaled out on demand, a stateless YSQL tier makes it easy to add nodes on demand. This enables rapid scaling of the cluster when more resources (CPU, memory, storage capacity) are required.

Highly Resilient w/ Native Failover & Repair

The underlying DocDB cluster is fault-tolerant. This means that node failures do not affect the SQL application using this distributed SQL database. It simply starts communicating to a new node as opposed to native PostgreSQL where the common approach of master-slave replication inevitably leads to manual failover and/or inability to serve recent commits.

Geo-Distribution w/ Multi-Region Deployments

DocDB supports geo-distributed deployments, meaning you can deploy a distributed SQL database across different geographic regions and zone.

Cloud Native Operations

DocDB allows dynamically changing nodes of the database with no app impact. Schema changes as well as infrastructure migrations are now zero downtime, even for a SQL database.

Summary

Bringing together two iconic database technologies such as Spanner and PostgreSQL into a new open source, cloud native database has been an immensely satisfying engineering achievement. However, we understand that a well-engineered database on its own right does not build trust in the minds of developers and architects. We have to earn that trust using the traditional means of communication, collaboration and sharing of success stories.

Through this series of posts, we explain our design principles, the tradeoffs associated with those principles, the actual implementation details and finally, the lessons learned especially around some of the more challenging aspects. We intend to prove our claims through exhaustive correctness testing (such as Jepsen) as well as comprehensive performance benchmarking (including TPCC). As we make rapid progress towards YSQL GA this summer, we are working closely with a few of our current users to highlight how YSQL can complement their existing investment in YugaByte DB. If your project can benefit from YSQL as well, don’t hesitate to reach us on our community Slack channel.

What’s Next?

• Compare YugaByte DB in depth to databases like CockroachDB, Google Cloud Spanner and MongoDB.
• Get started with YugaByte DB on macOS, Linux, Docker and Kubernetes.
• Contact us to learn more about licensing, pricing or to schedule a technical overview.

Django基础(10): URL重定向的HttpResponseDirect, redirect和reverse的用法详解

利用django开发web应用, 我们经常需要进行URL重定向，有时候还需要给URL传递额外的参数。比如用户添加文章完成后需要转到文章列表或某篇文章详情。因此熟练掌握HttpResponseDirect, redirect和reverse这三种方法对于Django Web开发是至关重要。今天小编我就带你看下Django URL重定向的HttpResponseDirect, redirect和reverse方法的详细用法。

HttpResponseDirect方法

HttpResponseRedirect是django首选的URL重定向方法，在django.http模块里。该方法的第一个参数是必要的，是用来重定向的URL地址。这个URL可以是完整的链接（比如’http://www.baidu.com‘），也可以是一个不包含域名的静态链接（例如‘/index/’）。

我们下面以新闻博客(blog)为例来看看如何使用HttpResponseDirect方法。假如我们有如下3个urls, 一个展示文章，一个添加文章，一个展示文章详情。我们需要使用该方法在视图中实现两种URL重定向:

转向不含参数的URL: 用户添加文章完成后转向文章列表(/index/); 或

转向包含参数的URL: 用户添加文章完成后转向文章详情(/article/2/new-day/)

from django.urls import path, re_path
from . import views

# namespace
app_name = ''blog''
urlpatterns = [
   # 展示所有文章
   path(''/index/'', views.ArticleListView.as_view(), name=''article_list''),
   # 展示文章详情
   re_path(r''^article/(?P<pk>\d+)/(?P<slug1>[-\w]+)/$'',
           views.ArticleDetailView.as_view(), name=''article_detail''),
   # 添加文章
   re_path(r''^article/create/$'',
           views.ArticleCreateView.as_view(), name=''article_create''),
]
 

1. 在视图views.py中利用HttpResponse重新定向至不含参数的URL

from .models import Article,
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import ArticleForm

def article_create(request):
   if request.method == ''POST'':
       form = ArticleForm(request.POST)
       if form.is_valid():
           form.save()  
           return HttpResponseRedirect("/index/")
   else:
       form = ArticleForm()
   return render(request, ''blog/article_create_form.html'', {''form'': form})
如果/index/页面有分页功能， 你还可以通过使用HttpResponseRedirect(''/index/?page=2'')直接获取第2页的文章列表。

HttpReponseDirect只支持hard coded urls(硬编码链接), 不能直接使用命名的URL，如使用HttpResponseDirect(''blog:article_list‘)是错误的。在使用URL命名时，我们需要先通过URL反向解析方法reverse先对命名URL(article_list)进行解析，然后再使用HttpReponseRedirect定向(如下面的代码)。背后的逻辑是reverse(''blog:article_list'')=''/index/''。

......
from django.http import HttpResponseRedirect
from django.urls import reverse
.....

def article_create(request):
   if request.method == ''POST'':
       form = ArticleForm(request.POST)
       if form.is_valid():
           form.save()
           return HttpResponseRedirect(reverse(''blog:article_list''))
  ....
 

2. 在视图views.py中利用HttpResponseDirect重新定向至包含参数的URL

对于包含参数的URL(/article/2/new-day/)，使用HttpResponseDirect定向前一般需要先使用reverse方法对命名的URL(如''article_detail'')进行解析，同时传递参数(如id, slug等）。

from .models import Article
from django.http import HttpResponseRedirect
from django.urls import reverse
from django.shortcuts import render
from .forms import ArticleForm

def article_create(request):
   if request.method == ''POST'':
       form = ArticleForm(request.POST)
       if form.is_valid():
           article = form.save()
           return HttpResponseRedirect(reverse(''blog:article_detail'', args=[str(article.pk), article.slug]))
   else:
       form = ArticleForm()
   return render(request, ''blog/article_create_form.html'', {''form'': form})
 

其中最重要的一行代码如下所示。大家可以仔细看下参数是如何传递到url的。

reverse(''blog:article_detail'', args=[str(article.pk), article.slug]
 

redirect方法

redirect是URL重新定向的便捷方法，在django.shortcuts模块里。HttpResponseRedirect能支持的URL重定向，redirect都支持。比如下面3种重定向是redirect的常规用法。

from django.shortcuts import redirect
from django.urls import reverse

# 案例1
def my_view(request):
   ...
   return redirect(''/index/'')

# 案例2
def my_view(request):
   ...
   return redirect(''https://www.baidu.com/'')

# 案例3
def my_view(request):
   ...
   return redirect(reverse(''blog:article_list''))
redirect真正NB的地方在于，它不仅能根据URL重定向，还可以根据对象Object重定向和根据视图view重定向，根据视图重定向的时候还可以传递额外的参数。

1. 根据对象Object重定向

使用该方法的前提是模型里已经定义了get_asbolute_url方法，使用redirect会自动调用get_absolute_url方法。

from django.shortcuts import redirect

def my_view(request):
   ...
   obj = MyModel.objects.get(...)
   return redirect(obj)
 

2. 根据视图view重定向

使用该方法的前提已对URL进行了命名，且对应了相应的视图。下面案例中redirect会先根据视图函数的名字查找对应url，在传递额外参数。后台工作还是由reverse方法来完成的。

def my_view(request):
   ...
   return redirect(''some-view-name'', foo=''bar'')
 

reverse方法

reverse方法的作用是对已命名的URL进行反向解析，还传递相应的参数（args或带key的参数kargs)。该方法位于django.urls模块。reverse方法一般有2种应用场景:

在模型中自定义get_absolute_url时使用，传递参数

在视图中对命名URL进行解析，传递参数，再使用HttpResponseDirect和redict进行重定向

1. 模型中自定义get_absolute_url，并传递参数args

def get_absolute_url(self):
   return reverse(''blog:article_detail'', args=[str(self.pk), self.slug])
 

2. 在视图中配合URL重定向使用，并传递kargs

from django.urls import reverse
from django.shortcuts import redirect

def my_view(request):
   ...
   return redirect(reverse(''admin:app_list'', kwargs={''app_label'': ''auth''}))
 

还有一点容易被人们忽略的是reverse方法不仅能对命名的urls进行反向解析，还可以对视图函数进行反向解析，找到需要重新定向的url, 如下面代码所示。当然但这个方法我们并不推荐使用。与此功能相反的是resolve方法，该方法可以根据url找到对应视图函数。

from django.urls import reverse
from blog import views
reverse(views.index)
 

小结

HttpResponseDirect, redirect和reverse三个方法都非常常用，在使用它们前忘了import进来哦。注意它们在不同的模块。

HttpResponseDirect - django.http

redirect - django.shortcuts

reverse - django.urls

今天关于GoogleredirectURI不允许使用片段url的解决方法和谷歌浏览器不允许加载本地资源的分享就到这里，希望大家有所收获，若想了解更多关于AngularJS Google Oauth 400错误redirect_uri的参数值无效：不允许使用片段、Apache CGIredirect到绝对URI不起作用、Distributed PostgreSQL on a Google Spanner Architecture – Query Layer、Django基础(10): URL重定向的HttpResponseDirect, redirect和reverse的用法详解等相关知识，可以在本站进行查询。