News Contents

Toplam 28 İlan Bulundu

Understanding the meaning of persistence is important for evaluating different data store systems. Given the importance of the data store in most modern applications, making a poorly informed choice could mean substantial downtime or loss of data. In this post, we‘ll discuss persistence and data store design approaches and provide some background on these in the context of Cassandra.

Persistence is "the continuance of an effect after its cause is removed". In the context of storing data in a computer system, this means that the data survives after the process with which it was created has ended. In other words, for a data store to be considered persistent, it must write to non-volatile storage.

If you need persistence in your data store, then you need to also understand the four main design approaches that a data store can take and how (or if) these designs provide persistence:

  • Pure in-memory, no persistence at all, such as memcached or Scalaris
  • In-memory with periodic snapshots, such as Oracle Coherence or Redis
  • Disk-based with update-in-place writes, such as MySQL ISAM or MongoDB
  • Commitlog-based, such as all traditional OLTP databases (Oracle,
    SQL Server, etc.)

In-memory approaches can achieve blazing speed, but at the cost of being limited to a relatively small data set. Most workloads have relatively small "hot" (active) subset of their total data; systems that require the whole dataset to fit in memory rather than just the active part are fine for caches but a bad fit for most other applications. Because the data is in memory only, it will not survive process termination. Therefore these types of data stores are not considered persistent.

The easiest way to add persistence to an in-memory system is with periodic snapshots to disk at a configurable interval. Thus, you can lose up to that interval‘s worth of updates.

Update-in-place and commitlog-based systems store to non-volatile memory immediately, but only commitlog-based persistence provides Durability -- the D in ACID -- with every write persisted before success is returned to the client.

Cassandra implements a commit-log based persistence design, but at the same time provides for tunable levels of durability. This allows you do decide what the right trade off is between safety and performance. You can choose, for each write operation, to wait for that update to be buffered to memory, written to disk on a single machine, written to disk on multiple machines, or even written to disk on multiple machines in different data centers. Or, you can choose to accept writes a quickly as possible, acknowledging their receipt immediately before they have even been fully deserialized from the network.

At the end of the day, you‘re the only one who knows what the right performance/durability trade off is for your data. Making an informed decision on data store technologies is critical to addressing this tradeoff on your terms. Because Cassandra provides such tuneability, it is a logical choice for systems with a need for a durable, performant data store.

Show More Devamı
Tüm Yorumlar - All Comments


Black-Box Testi:
Sadece fonksiyonellik ve ihtiyaçlara yönelik test çeşididir.  Black-box  testleri,   Closed Box,    Fonksiyonellik  testi,veya   Opaque  olarak da adlandırılır.  Black-Box  testleri, yapılacak işlemin fonksiyonelliğine uygun girdi(veri) seçimini ve bu girdinin yapılacak olan işlemin beklenen ya da beklenmeyen çıktı vermesini dikkate alır.

Black-Box  testleri 2 başlık altında incelenebilir.

Kullanıcının Var Olmadığı Testler:

  • Araştırma Testi : Test edilecek olan işlem ya da uygulamanın test öncesinde öğrenilmesi, hakkında fikir edinilmesini amaçlar.
  • Ad-Hoc Testi :  Genellikle yapılacak bütün testlerin sürelerinin belirlenmesini amaçlar.
  • Yenilenme Testi :Uygulamanın herhangi bir hataya karşı ne kadar sürede eski haline geleceğini test eder.Sistem gereksinimlerine göre,tip ve yenileme hızı belirlenir.
  • Seviye Testi : İşlemlerin yapılacağı sistemin uç noktasına kadar zorlanması ve bu uç noktanın belirlenmesi amaçlanır. Seviye Testi : İşlemlerin yapılacağı sistemin uç noktasına kadar zorlanması ve bu uç noktanın belirlenmesi amaçlanır.
  • Fonksiyonellik Testi(FunctionalTesting):  Adından da anlaşılacağı gibi uygulamaya ait fonksiyonların test edilmesidir. Beklenen çıktı alındığı sürece, test devam eder.
  • Kullanılabilirlik Testi(Usability Testing):  Özellikle arayüzünü bir çok kullanıcının kullanacağı ve arayüzün önemli olduğu uygulamalar için yapılması gereken bir test çeşididir.
  • Stres Testi(Stress Testing) : Aynı işlemin(Örn: Üye girişi) aynı anda yüksek adetlerde yapılması ve sonuçlarının izlenmesi olarak tanımlanabilir . Bu testte, yapılan işlemler karşısında sistemin verdiği tepki ve sistemin dayanıklılığı gözlenir.
  • Yükleme Testi(Load Testing):  Sistem performanslarını ölçmek amacı ile yapılır. Burada amaç, kullanılan sistemin nerede ve hangi koşullarda çökeceğini belirlemektir. 
  • Duman Testi(SmokeTesting) :  Genel olarak mantık testi olarak bilinir. Duman testinden geçmiş bir uygulama, daha büyük testler için hazırdır. Yani küçük testler sorunsuz yapılmıştır.

Kullanıcının Gerekli Olduğu Testler:
  • Kabul Testi(Acceptance Testing):  Kullanıcıların sistemin beklentileri karşılayıp karşılamadığına yönelik yaptığı testlerdir.
  • Alfa Testi(Alpha Testing):  Kullanıcıların uygulama geliştirme merkezine çağrıldığı ve kullanıcının burada yapacağı işlemlere göre geliştiricilerin düzeltilmesi/değiştirilmesi gereken kısımları not aldığı testlerdir. 
  • Beta Testi(Beta Testing):  Uygulamanın kullanıcılar tarafından test edildiği bir test çeşididir. Kullanıcılar için beta versiyonu yayınlanır. Ardından kullanıcılardan gelecek geri dönüşler doğrultusunda düzeltmeler yapılır.

Show More Devamı
Tüm Yorumlar - All Comments
Yusuf 15-06-2019 20:31
Yararlı bilgiler teşekkürler


Remember

  1. Multiprogramming – A computer running more than one program at a time (like running Excel and Firefox simultaneously).
  2. Multiprocessing – A computer using more than one CPU at a time.
  3. Multitasking – Tasks sharing a common resource (like 1 CPU).
  4. Multithreading is an extension of multitasking.


async: Do this by yourself somewhere else and notify me when you complete(callback). By the time i can continue to do my thing.

enter image description here

parallel: Hire as many guys(threads) as you wish and split the job to them to complete quicker and let me know(callback) when you complete. By the time i might continue to do my other stuff.

enter image description here

the main difference is parallelism mostly depends on hardware.

Show More Devamı
Tüm Yorumlar - All Comments

Backend-For-Frontend using GraphQL under Microservices

Microservices are an architectural style, in which the analysis and modeling of specific business areas, complex applications are decomposed into a group of services that are small and specific, have a low degree of coupling. Each microservice is a small, independent and deployable application.

This concept of microservice architecture is similar to the concept of separating the frontend and backend. The frontend application controls all of its own UI-level logic, while the business logic is accomplished through API calls to the relevant microservice. After the continuous growth of frontend single page application frameworks, and the spread of mobile apps, whether it is a Mobile Native development or a Hybrid application development, so frontend and backend separation has made Web and Mobile applications become clients. The client only needs to query and modify the resources through the API.

The following figures shows the main difference between the absence of BFF and the addition of BFF on the front and back ends.

1. Traditional frontend and backend separation application

 

In the traditional frontend and backend design, usually the Mobile App or Website directly accesses the backend service, the backend microservices call each other, and then return the final result to the frontend client. For clients (especially mobile), too many HTTP requests are expensive. Therefore, in order to minimize the number of requests, the frontend generally tends to obtain associated data through a single API. It means that sometimes the backend will do some UI-related logic processing to meet the needs of the client .

2. BFF’s frontend and backend architecture

 

The biggest difference in BFF’s frontend and backend architectures is that the frontend (Mobile, Web) no longer directly accesses the backend microservices, but rather accesses it through the BFF layer. And each client will have a BFF service. With BFF, there are fewer mutual calls between microservices. This is because some UI logic is processed at the BFF level.

BFF and API Gateway

From the above understanding of BFF, since BFF is a middle-tier service accessed by frontend and backend, what is the difference between BFF and API Gateway? Let’s first look at the common implementation of API Gateway. (The API Gateway may be designed in many ways. Here are just three examples.)

1. API Gateway provides the same API for all clients

For example: /api/accounts. In this case, API Gateway does not distinguish between client types, as shown below:

 

2. API Gateway provides separate APIs for each client

For example: /services/mobile/api/accounts and /services/web/api/accounts. The API Gateway determines which client it comes from, based on different APIs, and then processes them separately to return the resources required by different clients.

 

3. Multiple API Gateways provide separate APIs for each client

BFF is actually considered like this implementation mode of API Gateway.

 

GraphQL and REST

GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

What are the similarities and differences between GraphQL and REST? We can understand it by the following example:

REST can be accessed through the following way:

Request:

GET http://127.0.0.1/api/accounts

Response:

[
{
"id": 88,
"name": "Mena Meseha",
"photo": "
http://bucket.s3.amazonaws.com/photo.jpg"
},
...
]

For the same request if accessed with GraphQL, the process is as follows:

Request:

POST http://127.0.0.1/graphql

Body:

query {accounts { id, name, photo } }

Response:

{
"data": {
"accounts": [
{
"id": 88,
"name": "Mena Meseha",
"photo": "
http://bucket.s3.amazonaws.com/photo.jpg"
},
...
]
}
}

Compared to the REST style, GraphQL has the following features:

1. On demand data model definition

For example, for the above /accounts access, if the client is only interested in account.id and account.name. So client only need to pass account {id
name}
to query. The model is defined in the background. The client only needs to obtain the data it cares about.

2. Get many resources in a single request

For example, you may need to obtain account.friends and account.friends.address, In fact, this query need to get the data from 3 different resources which are accountfriends and address. In REST mode, the account data may needs to be sent again to another API to get the friends and address info. GraphQL can query for nested levels of data so it greatly reduces the number of client requests and allows the client to obtain all the required data through a single API as the following query:

{
account(id:88) { // 1st Level
name,
friends { // 2nd Level
name,
address { // 3rd Level
country,
city
}
}
}
}

GraphQL vs REST

For the comparison between GraphQL and REST, there are mainly the following differences:

1. Data Acquisition: REST lacks scalability and GraphQL can be accessed on demand. The payload can be extended when the GraphQL API is called.

2. API calls: REST’s operation for each resource is an endpoint, and GraphQL only needs a single endpoint, but the post body is not the same.

3. Complex data requests: REST requires multiple calls for nested complex data, GraphQL calls once, reducing network overhead.

4. Error code processing: REST can accurately return HTTP error code, GraphQL returns 200 uniformly, and wraps error information.

5. Version number: REST is implemented via v1/v2, and GraphQL is implemented through the Schema extension.

Microservices + GraphQL + BFF Practice

I recommend to use express-graphql to build the BFF side of the project and then deploy it through Docker, and between the BFF and the microservices backend, there is service registration and discovery are performed through Consul.

 
Sample Technical Architecture for Microservices + GraphQL + BFF
 
BFF Technology Stack

Authentication and Authorization

Authentication and authorization are other issue that need to be considered when using GraphQL. Do we process them before or after the GraphQL parsing process?

To answer this question, you can think of GraphQL as a DSL (Domain Specific Language) on top of your own backend data acquisition logic. We just need to think of it as an intermediate layer that can be placed between the client and our actual data service (or multiple services).

Then consider authentication and authorization as another layer. GraphQL is not useful in the actual implementation of authentication or authorization logic because it does not mean it. However, if we want to place these layers behind GraphQL, we can use GraphQL to pass the access token between the client and Auth API. This is very similar to the way we authenticate and authorize via the RESTful API.

3. GraphQL + BFF Practices

Through the following aspects, you can think about some of the better qualities of GraphQL-based BFF:

GraphQL and BFF’s focus on business points

 

From a business point of view, the property manager (PM App) is concerned with property, because the property manager manages a group of houses, so it is necessary to know all the house profiles, and for each house need to know whether there is a corresponding maintenance application. Therefore, PM App BFF data structure is defined, maintemamceRequests is property sub-attribute.

With similar data, the house maintenance supplier (Supplier App) is concerned with maintenance request (maintenance work order), so in the data obtained by the Supplier App, our main body is maintenanceRequest.

So because there are different usage scenarios, different clients have different concerns for the same data. From this perspective, the data structure defined in the BFF is what the client really cares about. BFF is born for the client and is part of the client. It should be noted that the "business focus" does not mean that the BFF will handle all the business logic. The business logic should still be taken care of by the micro service. The BFF is concerned with what the client needs.

GraphQL support for versioning

 
Figure (1) -> Figure (2) -> Figure (3)

Assume BFF end has been released into the production environment. Now we need to change the structure of Figure (1) to the structure of Figure (2), but in order not to affect the API access of the old users, our BFF API must be compatible at this time. If REST, we may update the API version. But in BFF, we can use the structure of Figure (3) for forward compatibility. At this time, the old APP uses the data structure of the yellow area, while the new APP uses the structure defined by the blue area.

Conclusion

Building a BFF based on GraphQL under microservices is not a silver bullet. It is not necessarily suitable for all projects. For example, after you use GraphQL, you may have to face multiple query performance issues, but this does not prevent it from becoming a good attempt. You do see that Facebook has already used GraphQL, and that Github has also opened GraphQL’s API. This article only serves as an introduction to the idea. I hope that there will be more sharing about the use of BFF in the community.

Show More Devamı
Tüm Yorumlar - All Comments